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SUMMARY 


The  Training  Decisions  System  (TDS)  is  a  computer-based  decision 
support  technology  which  has  been  developed  to  provide  a  more  integrated 
approach  to  Air  Force  training  planning  and  programming.  The  present  TDS 
technology  has  been  designed  and  developed  to  address  the  what  (training 
content),  the  where  (classroom,  hands-on,  self  study,  and  on-the-job 
training),  and  the  when  (at  what  point  in  an  airman's  career)  of  specialty 
training.  The  TDS  identifies  specialty  jobs  and  training  programs  in 
terms  of  groups  of  related  tasks  (Task  Training  Modules),  examines  the 
allocation  of  task  modules  to  various  training  settings,  and  estimates 
training  capacities  and  costs  for  representative  organizational  units. 
Further,  the  TDS  incorporates  modeling  strategies  to  allow  functional  or 
training  managers  to  conceptualize  present  utilization  and  training 
patterns  and  to  ask  "what  If"  questions  involving  possible  future  policy 
changes  within  the  Air  Force  functional  and  training  environments.  Such 
policy  alternatives  can  be  evaluated  by  Air  Force  decision  makers  in  terms 
of  their  relative  costs  and  constraints  on  specialty  training  which  can  be 
identified  for  all  training  settings.  The  present  TDS  technology  consists 
of  three  major  data-based  subsystems  and  a  fourth  integration  and 
optimization  subsystem,  with  supporting  software.  This  report  provides  a 
brief  overview  of  the  TDS  research  program  conducted  on  four  Air  Force 
specialties  (AFSs):  Electronic  Computer  and  Switching  Systems  (AFS  305X4), 
Avionic  Inertial  and  Radar  Navigation  Systems  (AFS  328X4),  Aircraft 
Environmental  Systems  (AFS  423X1),  and  Security  Police/Law  Enforcement 
(AFS  811XX) .  In  addition,  a  summary  of  the  objective  data  sources  and 
data  collection  activities  which  are  needed  to  form  the  decision  support 
functions  is  provided.  The  report  also  examines  issues  which  arose  during 
the  course  of  this  project  in  terms  of  recent  trends  in  Air  Force 
personnel  and  training  policy  that  affect  training  planning.  Finally,  the 
report  discusses  recommendations  for  future  training  decisions  technology 
research  and  development  and  for  using  TDS  research  outcomes  in  the 
operational  Air  Force. 
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PREFACE 


The  Training  Decisions  System  (TDS)  is  a  multi-year  exploratory 
research  and  development  effort  by  the  Air  Force  Human  Resources 
Laboratory  sponsored  by  HQ  USAF  and  HQ  ATC.  The  goal  of  this  effort  is 
to  develop  a  proof -of -concept,  computer-based  decision  support 
technology  for  a  more  integrated  approach  to  Air  Force  training 
planning  and  programming. 

The  initial  TDS  has  been  developed  for  the  United  States  Air  Force 
under  Contract  Number  F33615-83-C-0028.  This  initial  proof -of -concept 
effort  was  accomplished  by  the  Human  Factors  section  of  the  Systems 
Engineering  and  Analysis  Department,  McDonnell  Douglas  Astronautics 
Company,  St.  Louis,  Missouri,  and  CONSAD  Research  Corporation, 
Pittsburgh,  Pennsylvania.  In  the  initial  phases  of  the  work,  the 
Research  Triangle  Institute,  Research  Triangle  Park,  North  Carolina, 
assisted  in  the  project.  In  the  final  phase  of  the  TDS  project,  the 
Manpower,  Personnel,  Training,  and  Research  Division  of  the  MAXIMA 
Corporation,  San  Antonio,  Texas,  assisted  in  the  sensitivity  analyses. 

This  volume  is  the  final  report  of  the  initial  exploratory  research 
and  development  effort  and,  as  such,  provides  a  general  overview  of  the 
design  work  performed  to  develop  the  proof -of -concept  TDS.  Details  of 
each  phase  of  the  effort  can  be  found  in  the  TDS  reports  noted  below: 

Vaughan,  D.S.,  Yadrick,  R.M.,  Perrin,  B.M.,  Cooley,  P.C.,  Dunteman, 

G.H.,  Clark,  B.L. ,  &  Rueter,  F.H.  (1984,  August).  Training 

decisions  system  preliminary  design.  Unpublished  manuscript. 

Vaughan,  D.S.,  Yadrick,  R.M.,  Perrin,  B.M.,  Mitchell,  J.L. ,  Sturdevant, 
W.S.,  Rueter,  F.H.,  &  Ward,  J.,  Jr.  (1985,  September).  Training 
decisions  system  transition  plan.  Unpublished  manuscript. 

Perrin,  B.M.,  Knight,  J.R.,  Mitchell,  J.L. ,  Vaughan,  D.S.,  &  Yadrick, 
R.M.  (1988,  September).  Training  decisions  system:  Development  of 
the  task  characteristics  subsystem  (AFHRL-TR-88-15,  AD-A199  094). 
Brooks  AFB,  TX:  Training  Systems  Division,  Air  Force  Buman 
Resources  Laboratory. 

Yadrick,  R.M.,  Knight,  J.R. ,  Mitchell,  J.L. ,  Vaughan,  D.S.,  &  Perrin, 
B.M.  (1988,  July).  Training  decisions  system:  Development  of  the 
field  utilization  subsystem  ( AFHRL-TR-88-7 ,  AD-A198  087 Brooks 
AFB,  TX:  Training  Systems  Division,  Air  Force  Human  Resources 
Laboratory. 

Rueter,  F.H.,  Feldsott,  S.I.,  &  Vaughan,  D.S.  (1989).  Training 

decisions  system:  Development  of  the  resource  cost  subsystem 
(AFHRL-TR-88-52) .  Brooks  AFB,  TX:  Training  Systems  Division,  Air 
Force  Human  Resources  Laboratory. 

Vaughan,  D.S.,  Mitchell,  J.L. ,  Marshall,  G.A.,  Feldsott,  S.I.,  & 

Rueter,  F.H.  (1988,  August).  Training  decisions  system  procedural 
guide:  TDS  user  instructions.  Unpublished  manuscript . 
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RESEARCH  AND  DEVELOPMENT  OF 
THE  TRAINING  DECISIONS  SYSTEM 


1.  INTRODUCTION 

The  Training  Decisions  System  (TDS)  is  designed  to  assist  Air  Force 
managers  in  making  critical  judgments  concerning  the  what,  when,  and  where 
of  training  required  for  an  Air  Force  specialty  (AFS)  by  providing  models 
which  characterize  the  current  and  several  alternative  Utilization  and 
Training  patterns,  and  methods  for  evaluating  these  models.  Using  the 
TDS,  Air  Force  managers  can  assess  the  relative  impact  of  each  proposed 
alternative  in  terms  of  relevant  costs  and  resource  requirements, 
including  the  costs  of  on-the-job  training  (OJT).  The  TDS  can  help 
specialty  managers  formulate  optimal  (cost  effective)  manpower,  personnel, 
and  training  (MPT)  policies  for  the  specialty. 


1.1  Technical  Approach 

The  TDS  was  a  proof-of-concept  project;  the  initial  phase  of  the 
effort  involved  the  collection  and  analysis  of  information  concerning 
present  Air  Force  training  decision  making.  The  types  of  data  needed  by 
Air  Force  managers  were  identified  as  well  as  ways  to  collect  or  develop 
information  not  previously  readily  available  to  decision  makers.  A  top- 
down  structured  analysis  approach  was  then  used  to  design  system 
components  to  identify  groups  of  tasks  which  could  be  trained  together;  to 
model  the  specialty  in  terms  of  jobs  and  training  requirements;  and  to 
collect,  analyze,  or  predict  needed  data  and  functions.  The  TDS  design 
included  integration  of  components  into  an  overall  system  which  could 
display  data  to  properly  assist  Air  Force  decision  makers.  Software  was 
developed  for  the  TDS  to  operate  on  the  AFHRL  UNISYS  1100  Computer. 


1.2  Innovations 

The  TDS  project  required  the  development  of  innovative  new  approaches 
for  modeling  the  dynamics  of  military  occupations  and  assessing  job 
training  requirements.  Allocation  curves  were  developed  to  relate  hours 
of  training  in  different  settings  (such  as  classroom,  hands-on,  self 
study,  and  OJT)  to  the  degree  of  proficiency  which  can  be  achieved  for 
groups  of  related  job  tasks.  The  specialty  modeling  approach  was 
integrated  with  state-of-the-art  econometric  techniques  to  estimate  total 
training  capacities  and  costs  in  different  training  settings,  including 
OJT.  The  TDS  was  designed  to  make  maximum  possible  use  of  existing  data 
bases;  it  also  required  development  of  data  collection  methodologies  and 
creation  of  new  techniques  to  estimate  job  and  training  assignment 
probabilities  at  various  career  stages.  Computer  programs  were  created  to 
simulate  the  complex  flows  of  people  through  jobs  and  training  programs. 


1.3  Applications  of  the  TDS 

The  TDS  was  developed  for  use  by  members  of  the  Air  Staff  to  support 
the  Planning,  Programming,  and  Budgeting  System  (PPBS);  functional 
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managers  and  major  command  (MAJCOM)  training  staffs  to  evaluate  proposals 
for  changing  specialty  structure  or  training  programs;  Training  Planning 
Teams  (TPTs)  in  managing  major  changes  in  AFS  training  (AFR  50-8);  and 
Utilization  and  Training  Workshops  (U  &  TWs)  convened  by  the  Air  Training 
Command  (HQ  ATC)  to  plan  implementation  of  training  changes.  HQ  ATC  and 
its  subordinate  units  responsible  for  Instructional  Systems  Development 
(ISD)  can  make  use  of  the  system  for  identifying  Air  Force  specialty 
training  requirements  ("front  end  analysis")  and  to  assess  the  cost  of 
proposed  changes  in  training. 

Other  Manpower,  Personnel,  and  Training  (MPT)  agencies,  such  as  the 
Air  Force  Military  Personnel  Center  (AFMPC) ,  Air  Force  Management 
Engineering  Agency  (AFMEA),  and  the  USAF  Occupational  Measurement  Center 
(LBAFOMC)  can  use  the  TDS  to  assess  the  impact  of  proposed  AFS  changes  or 
the  reengineering  of  jobs.  The  MPT  research  community  could  employ  the 
TDS  to  secure  needed  AFS-specific  data,  such  as  training  requirements  or 
costs,  or  may  use  the  system  as  a  foundation  for  studying  MPT  integration. 


1.4  Organization  of  Report 

This  final  technical  report  summarizes  the  work  done  on  the  system, 
provides  an  overview  of  the  system  developed,  notes  problem  areas, 
suggests  areas  for  further  research,  and  provides  recommendations  for  the 
operational  implementation  of  the  system.  The  report  is  organized  as 
follows. 

Chapter  2  provides  a  conceptual  overview  of  the  TDS  developed  in  this 
research  and  development  ( R&D )  project.  The  background  of  the  project  is 
discussed,  along  with  the  recent  trends  in  Air  Force  training  decision 
making  and  previous  AFHRL  programs  which  have  led  to  the  present  line  of 
research.  The  complex  requirements  for  various  types  of  objective  data 
for  use  in  Air  Force  training  decision  making  are  discussed,  and  how  such 
needs  were  met  in  the  design  of  TDS  subsystems  are  noted.  References  are 
provided  for  those  interested  in  more  detail  on  the  system  or  the  various 
phases  of  its  development. 

Chapter  3  briefly  reviews  the  agencies  which  might  make  use  of  the  TDS 
and  the  various  ways  in  which  the  system  can  be  employed.  Ways  in  which 
users  can  interface  with  the  present  prototype  system  are  outlined,  as 
well  as  suggestions  for  how  such  user  interface  should  be  implemented  in 
an  operational  system. 

Chapter  4  notes  some  of  the  issues  raised  and  problems  encountered  in 
conducting  the  project.  Specific  subsystem  issues  are  outlined  and 
recommendations  made  for  future  R&D  work  to  resolve  these  issues. 

Chapter  5  discusses  several  conceptual  issues  in  current  Air  Force 
training  decision  processes  and  recommends  ways  in  which  these  might  be 
resolved.  The  implementation  of  TDS  as  an  operational  system  is  recom¬ 
mended  along  with  parallel  R&D  efforts  to  cross- validate  the  system  with 
studies  of  additional  specialties  and  develop  enhancements  to  be 
incorporated  into  the  system.  Suggestions  for  specific  Air  Force 
organizational  responsibilities  to  implement  the  system  are  outlined  in  a 
proposed  Air  Force  Regulation  (see  Appendix  A). 


2 


2.  A  CONCEPTUAL  OVERVIEW 


For  nearly  two  decades,  the  Air  Force  has  been  using  the  Instructional 
Systems  Development  (ISD)  model  to  guide  the  design  of  technical  training 
for  enlisted  occupations  and  for  support  of  new  weapon  system  acquisition 
(see  AFR  50-8).  The  ISD  model  requires  a  systems  approach  to  training 
development,  an  approach  aimed  at  providing  "optimal  training"  for  each 
specialty  or  weapon  system  (AFM  50-2,  AFP  50-58).  Many  of  the  data 
elements  required  for  the  application  of  ISD,  however,  are  not  readily 
available  in  existing  Air  Force  data  bases.  Further,  not  all  decision 
algorithms  suggested  in  the  ISD  model  have  been  empirically  validated  nor 
are  such  algorithms  equally  applicable  to  all  types  of  specialties. 
Decisions  involving  aircraft  maintenance  specialties,  for  example,  may 
require  an  approach  to  training  decision  making  that  addresses  issues 
which  have  a  direct  and  immediate  impact  on  combat  sortie  generation, 
whereas  for  a  support  specialty,  the  issues  may  involve  primarily  the  cost 
per  student  or  student  flow  limitations. 

Recent  developments  in  occupational  analysis  and  training  research,  as 
well  as  in  Air  Force  decision  making  processes  (see  AFR  50-8),  have 
created  new  opportunities  for  optimizing  training.  Such  developments 
include  the  recent  emergence  of  Utilization  and  Training  Workshops  (U  &  TWs) 
and  Training  Planning  Teams  (TPTs)  as  the  primary  vehicles  for  making 
major  training  decisions.  Such  innovative  procedural  changes  also  make 
obvious  a  need  for  a  technologically  advanced  data  generation,  analysis, 
and  evaluation  capability.  To  make  good  decisions  about  the  training 
needed  for  an  Air  Force  specialty  or  system,  decision  makers  must  be  able 
to  visualize  and  understand  the  jobs  and  training  programs  of  the 
specialty  or  weapon  system  under  consideration  and  its  technical  training 
and  Professional  Military  Education  (PME)  requirements,  as  well  as  the 
relative  costs  and  payoffs  of  various  training  options.  Such  a  "model" 
provides  a  concise  summary  of  the  current  status  of  the  specialty,  creates 
a  common  "language"  for  discussion  or  negotiation,  and  forms  the  baseline 
against  which  various  alternative  proposals  can  be  evaluated. 

To  provide  adequate  support  for  such  advanced  training  decision 
making,  the  Air  Force  Deputy  Chief  of  Staff  for  Personnel,  Education  and 
Training  (HQ  USAF/DPPE)  requested  that  the  Air  Force  Human  Resources 
Laboratory  develop  a  computer-based  Training  Decisions  System  (TDS)  to 
augment  the  Air  Force  ISD  model.  Such  a  system  would  generate  necessary 
front-end  training  requirements  data,  validated  decision  algorithms,  and 
procedures  for  improved  interaction  among  training,  personnel,  and 
functional  managers.  The  TDS  would  focus  on  supporting  Air  Force  managers 
in  making  decisions  as  to  the  what,  where,  and  when  of  the  training 
(including  the  On-the-Job  Training)  required  for  a  given  enlisted 
specialty  (Ruck,  1982). 


2.1  Background 

over  several  decades,  the  Air  Force  has  evolved  a  task-based  approach 
to  determining  technical  training  content  and  reviewing  personnel 
classification  and  utilization  policies  (Christal,  1974;  Mitchell,  1988; 
Morsh,  1964;  also  see  AFR  8-13).  As  part  of  the  occupational  analysis 
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(OA)  process,  tasks  are  defined  by  subject-matter  experts  (SMEs)  of  a 
specialty  in  their  own  technical  terminology,  working  with  analysts  of  the 
USAF  Occupational  Measurement  Center,  Randolph  Air  Force  Base,  Texas  (see 
AFR  35-2).  Several  kinds  of  data  on  these  tasks  are  collected  from  job 
incumbents  and  supervisors  for  use  in  reviewing  training  programs  (see 
ATCR  52-22).  large  samples  of  incumbents -are  asked  to  provide  information 
about  which  tasks  they  perform  in  their  present  jobs  and  the  relative 
amount  of  their  job  time  spent  performing  such  tasks.  These  data  are  used 
to  examine  the  variety  of  specialized  jobs  within  a  specialty  (occupa¬ 
tion),  to  assess  how  jobs  change  at  advanced  skill  levels,  and  to  review 
official  specialty  descriptions  and  initial  training  programs  (Christal  & 
Weissmuller,  1988;  Mitchell,  Ruck,  &  Driskill,  1988). 

One  of  the  most  important  data  elements  developed  during  the  OA 
process  involves  noncommissioned  officer  (NCO)  ratings  of  tasks  in  terms 
of  recommended  training  emphasis  for  first -term  and  first-job  airmen. 
Such  training  emphasis  (TE)  ratings  have  been  validated  empirically  using 
explanatory  regression  models  in  studies  of  18  AFSs  (Ruck,  Thompson,  & 
Stacy,  1987;  Ruck,  Thonpson,  &  Thomson,  1978;  Stacy,  Thompson,  &  Thomson, 
1977)  Two  important  findings  of  these  research  studies  were  that  super¬ 
visors  agreed  substantially  with  one  another  on  their  recommendations  in 
most  (but  not  all)  career  fields,  and  that  supervisors'  judgments  were 
explainable  in  terms  of  key  ISD  factors.  A  third  important  finding  was 
that  supervisors  could  not  agree  as  to  the  appropriate  sites  for  training 
technical  tasks.  TE  ratings  are  used  operationally  to  evaluate  course 
content  of  basic  technical  training  courses  for  first  enlistment  or  first 
job  personnel;  typically  they  are  not  used  to  evaluate  field  training 
detachment  (FTD)  or  mobile  training  team  (MTT)  courses  or  OJT  programs 
(Mitchell,  Ruck,  &  Driskill,  1988;  Mitchell,  Sturdevant,  Vaughan,  & 
Rueter,  1987;  see  also  ATCR  52-22).  Hence,  although  methods  had  been 
developed  and  validated  for  prioritizing  AFS  job  tasks  in  terms  of  recom¬ 
mended  training  emphasis  for  first  enlistment  personnel,  no  reliable  data 
were  yet  available  for  determining  appropriate  training  setting  and  site. 

By  1980,  the  determination  of  training  setting  was  being  made  at 
U  &  TWs,  where  trainers  and  training  managers  met  with  representatives  from 
operational  commands  to  negotiate  training  content  and  training  setting 
(Mitchell  et  al.,  1987;  see  also  ATCR  52-15).  These  conferences  grew  out 
of  earlier  procedures  developed  to  bring  initial  skills  technical  training 
in  line  with  initial  job  requirements  ("HASTY  GRAD"  projects),  while  at 
the  same  time  planning  for  those  training  requirements  deferred  to  FTD, 
MTT  or  OJT  (Ruck  &  Birdlebough,  1977;  Vaughan,  1978).  Only  minimal  data 
were  available  for  determining  appropriate  training  settings  for  specialty 
tasks;  thus,  these  decisions  were,  of  necessity,  based  almost  entirely 
upon  the  conferees'  personal  experience,  or  on  known  constraints  at  the 
resident  training  school.  For  these  reasons,  many  of  the  decisions  made 
in  U  &  TWs  cannot  be  consistently  replicated.  In  addition,  no  formal 
evaluation  or  estimates  were  made  of  the  impact  of  such  decisions  on 
personnel  utilization,  OJT  costs,  or  mission  performance  (Ruck,  1982). 


2.2  Approach 

The  general  strategy  used  in  developing  the  Training  Decisions  System 
focused  on  first  defining  functional  requirements  and  then  developing  the 
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structure,  methodologies,  and  procedures  for  meeting  those  requirements. 
The  research  team  collected  as  much  information  as  possible  concerning  Air 
Force  training  decisions  and  how  they  were  made;  a  variety  of  agencies  and 
offices  were  visited  to  define  the  desired  functions  and  data  requirements 
of  the  system  (Mitchell,  Sturdevant,  Vaughan,  &  Rueter,  1987). 

Based  on  the  information  gathered  from  such  visits  and  a  review  of  the 
previous  research  literature,  the  following  types  of  variables  were 
determined  to  be  particularly  important  for  Air  Force  decision  makers  and 
thus  must  be  represented  explicitly  within  the  TDS : 


•  Tasks  of  the  Specialty  and  Their  Associated  Characteristics 

•  Task  Allocations  to  Training  Settings 

•  Managers'  Preferences  for  Task  Allocations  to  Training  Settings 

•  Times  Required  to  Train  Tasks  in  Various  Setting  Allocations 

•  Utilization  and  Training  Patterns,  in  terms  of; 

-  Jobs  and  Associated  Tasks 

-  Training  States 

-  Transition  Probabilities  Among  Jobs  and  Training  States 

-  Numbers  of  Airmen  in  Various  Training  and  Job  States 

-  Airman  Proficiency  States 

•  Training  Costs 

•  Training  Resource  Requirements 

•  Training  Capacities 

•  Managers'  Preferences  for  U  &  T  Patterns 


Some  data  on  the  tasks,  jobs,  and  training  states  are  generally 
available  from  existing  sources,  such  as  the  occupational  analysis  program 
or  course  documentation.  For  other  variables,  few  if  any  data  are 
available  in  existing  Air  Force  data  bases.  For  example,  data  concerning 
OJT  costs  are  not  routinely  available,  and  innovative  approaches  were 
required  to  develop  such  information  for  use  by  training  decision  makers. 

A  preliminary  integrated  systems  design  was  developed  to  guide  the  R&D 
effort  which  provided  for  a  set  of  interactive  subsystems  to  deal  with  the 
three  broad  classes  of  data  (task  information,  training  and  job  patterns, 
and  costs  versus  training  capacity  constraints)  plus  a  fourth  subsystem  to 
integrate  such  data  and  display  the  data  for  decision  makers.  The 
preliminary  system  design  was  validated  and  refined  through  interactions 
with  potential  Air  Force  users  (Vaughan,  Yadrick,  Perrin,  Cooley,  Dunteman, 
Clark,  &  Rueter,  1984). 

The  general  approach  used  in  TDS  development  was  to  start  from  known 
data  bases,  such  as  the  OA  report  files  or  personnel  flow  statistics,  and 
to  develop  new  data  gathering  technologies  to  determine,  estimate,  or 
approximate  other  required  information.  Where  possible,  an  evaluation  of 
alternative  approaches  was  conducted;  several  methods  were  initially 
tested  and  the  method  yielding  the  best  results  was  adopted  for  use  in  the 
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TDS.  Where  necessary,  experimental  designs  were  employed  when  it  was 
necessary  to  verify  method  effects  or  analyze  data  differences.  General¬ 
ly,  data  collection  and  analysis  methods  were  developed  on  two  Air  Force 
specialties  (Avionic  Inertial  and  Radar  Navigation  Systems,  AFS  328X4,  and 
Security  and  Law  Enforcement,  AFS  811XX),  and  were  later  validated  and 
refined  on  two  additional  specialties  (Electronic  Computer  and  Switching 
Systems,  AFS  304X4,  and  Aircraft  Environmental  Systems,  AFS  423X1). 

Such  development  and  testing  was  done  in  a  systematic,  integrated  way 
to  ensure  that  the  various  types  of  information  could  be  synthesized  into 
a  coherent  picture  of  the  various  ways  training  requirements  of  a 
specialty  could  be  met  and  the  relative  cost  of  each  alternative.  The 
objective  was  to  make  visible  to  specialty  managers  and  training  staff 
officers  their  decision  options,  as  well  as  the  constraints  and  relative 
cost  consequences  of  each  decision. 

At  periodic  intervals  throughout  the  TDS  development  project,  the 
research  and  development  team  briefed  their  progress  and  results  to  an  Air 
Force  TDS  advisory  panel,  which  included  representatives  of  HQ  USAF,  HQ 
Air  Training  Command,  the  USAF  Occupational  Measurement  Center,  functional 
specialty  managers,  and  training  managers.  Such  periodic  progress  reviews 
were  extremely  constructive  in  terms  of  constructive  critique  of  results 
and  positive  interaction  between  researchers  and  potential  system  users. 
Suggestions  were  made  for  needed  design  improvements,  data  displays,  new 
data  sources  to  be  evaluated,  or  other  possible  solutions  to  research  and 
development  problems.  As  a  result  of  such  interactions,  a  separate  plan 
was  developed  to  guide  the  transition  of  TDS  from  a  research  and 
development  project  to  an  operational  system  in  an  appropriate  Air  Force 
organization;  this  transition  plan  was  circulated  to  various  Air  Force 
agencies  and  offices  for  coordination  and  staffing  (Vaughan,  Yadrick, 
Perrin,  Mitchell,  Sturdevant,  Rueter,  &  Ward,  1985).  The  end  result  of 
such  interactive  progress  reviews  was  an  improved  systems  design  and  TDS 
products  which  should  be  much  more  useful  to  potential  TDS  users. 


2.3  Results  -  The  Proof-of -Concept  Training  Decisions  System 

The  initial  research  and  development  of  the  TDS  has  been  completed. 
The  proof-of -concept  system  consists  of  three  interdependent  subsystems 
which  deal  with  Task  Characteristics,  Field  Utilization  pattern  modeling, 
and  Cost/Resource  data  generation,  as  well  as  a  fourth  Integration  and 
Optimization  subsystem  (see  Figure  1).  This  section  of  the  report  will 
present  a  conceptual  overview  of  the  system  to  provide  a  general 
perspective  of  the  TDS  and  a  basic  understanding  of  its  structure  and 
operation.  For  the  purposes  of  the  present  report,  a  short  description  of 
each  subsystem  and  how  it  was  developed  should  suffice. 

[For  the  details  of  the  preliminary  design,  see  Vaughan,  Yadrick, 
Perrin,  Cooley,  Dunteman,  Clark,  &  Rueter,  1984;  for  the  proposed 
transition  plan,  see  Vaughan,  Yadrick,  Perrin,  Mitchell,  Sturdevant, 
Rueter,  &  Ward,  1985.  For  details  of  the  development  of  each  subsystem, 
see  the  subsystem  technical  reports  cited  in  the  text  below  and  listed  in 
the  Reference  section.] 
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Figure  l.  Major  Subsystems  of  the  Training  Decisions  System. 


2.3.1  Task  Characteristics  Subsystem  (TCSj 

The  TCS  is  composed  of  two  components.  The  Task  Training  Module  (TTM) 
construction  component  is  designed  to  generate  TTMs  as  the  basic  unit  of 
analysis  in  the  TDS.  TTMs  solve  a  number  of  problems  associated  with  tne 
use  of  task-level  data  (see  Perrin,  Knight,  Mitchell,  Vaughan,  &  Yadrick, 
1987).  As  noted  earlier,  the  Air  Force  presently  makes  some  use  of 
task-based  data  for  training  decisions;  one  problem  is  that  many  basks 
share  a  common  skill  and  knowledge  base.  TTM-level  data  reflect  shared 
skills  and  knowledges,  thus  reducing  the  possibility  of  overestimating 
training  requirements.  A  second  problem  in  using  task  data  is  the  fact 
that  each  specialty  involves  300  to  2,000+  tasks,  far  too  many  for 
managers  to  process  in  a  typical  U  &  TW  session.  Indeed,  U  &  TW  participants 
SSSally  focus  on  review  of  the  relevant  Specialty  Training  Standard, 
leaving  detailed  review  of  tasks  to  occupational  analysts  and  training 
developers. 

According  to  some  interpretations  of  the  ISD  literature,  in  order  to 
fully  understand  and  classify  all  the  tasks  of  a  specialty  in  terms  of 
their  common  skills  and  knowledges,  a  detailed  task  analysis  would  be 
required  on  every  task  before  any  decisions  about  training  could  be  made. 
■The  fact  is  that  task  analysis  is  an  exceedingly  time-consuming,  labor- 
SlensTve  Spensive  P™,  and  various  t^es  of  tasks  »ay  require 
differing  types  of  analysis  (DeVries,  Eschenbrenner,  &  Ruck,  1980, 

Eschenbrenner,  DeVries,  Miller,  &  Ruck,  1980). aslf  ^alysis  for 
cannot  afford  the  manpower  and  expense  of  a  detailed  task  analysis  tor 

every  task  of  every  specialty. 
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Therefore,  a  procedure  is  needed  to  group  or  cluster  tasks  which  share 
common  skills  and  knowledges;  i.e.,  those  tasks  which  could  be  trained 
together  most  efficiently.  In  the  TDS,  individual  tasks  are  grouped  into 
clusters  of  related  tasks  called  Task  Training  Modules  (TTMs). 

2. 3. 1.1  TTM  Construction  Component;  Judgmental  Versus  Statistical 
Clusters.  Two  approaches  to  the  problem  of  constructing  TTMs  were 
evaluated  empirically  in  the  TDS  R&D  effort  (Perrin,  Vaughan,  Yadrick, 
Mitchell,  &  Knight,  1986).  One  was  a  judgmental  approach,  using  the 
expertise  of  SMEs?  a  second  approach  used  data  from  the  most  recent 
occupational  survey  to  cluster  tasks  statistically.  In  the  judgmental 
approach,  SMEs  were  asked  to  sort  the  tasks  of  their  specialty  into 
subjective  categories;  Le.,  to  group  those  tasks  which  should  be  trained 
together.  Presumably,  the  resulting  task  groupings  would:  (a)  contain 
tasks  which  shared  similar  underlying  skills  or  knowledges,  and  (b) 
include  tasks  performed  in  the  same  job.  It  is  a  "given"  that  the  same 
task  might  appear  in  more  than  one  job.  Results  from  the  attempts  to 
apply  this  judgmental  approach  in  the  initial  two  specialties  (Avionic 
Inertial  &  Radar  Navigation  Systems,  AFS  328X4  and  Security  &  Law 
Enforcement,  AFS  811XX)  were  not  encouraging,  since  each  group  of  SMEs 
tended  to  use  unique  conceptual  approaches  depending  on  their  backgrounds 
and  present  assignment.  Even  though  a  fair  consensus  was  achieved  through 
extended  negotiation,  a  complete  consensus  was  not  attained.  Attempts  to 
refine  the  clusters  with  new  reviewers  resulted  in  only  minor 
modifications  based  on  their  unique  perspectives  of  their  specialty. 

In  the  second  approach,  multivariate  statistical  techniques  were 
employed  in  an  iterative  manner  with  a  variety  of  input  data,  such  as  the 
probability  of  co-performance  of  tasks  calculated  from  available 
occupational  survey  report  (OSR)  data,  common  equipment  usage,  and  skill- 
level  or  grade  information.  Initial  trials  indicated  that  the  latter 
variables  added  little  or  no  refinement  to  statistical  clusters  formed 
using  only  a  measure  of  task  co-performance.  Advanced  work  focused  on  use 
of  Comprehensive  Occupational  Data  Analysis  Programs  (CODAP)  hierarchical 
clustering  of  tasks  and  interpretation  of  meaningful  clusters  of  tasks  as 
TTMs.  It  was  found  that  the  help  of  SMEs  was  critical  to  naming  the 
modules,  assessing  their  significance,  and  allocating  isolated  tasks  which 
did  not  statistically  cluster.  Thus,  the  final  recommended  procedure  was 
a  combined  approach  that  utilized  statistical  co-performance  clustering  to 
form  initial  groupings  of  tasks,  having  an  analyst  identify  task  groupings 
which  appeared  meaningful,  and  then  having  SMEs  title  and  refine  the  task 
groupings  into  TTMs.  This  approach  saves  time  for  both  analysts  and  SMEs 
and  provides  a  structured  focus  for  their  efforts.  Detailed  task  analysis 
of  the  tasks  of  a  sample  of  TTMs  for  two  specialties  indicated  the  tasks 
in  TTMs  formed  through  this  process  did  indeed  share  common  skill  and 
knowledge  requirements  (Perrin  et  al.,  1986). 

2. 3. 1.2  Training  Setting  Allocation  Component.  Once  TTMs  are 
finalized,  survey  instruments  are  developed  with  which  to  gather 
information  as  to  how  TTMs  are  and  should  be  allocated  to  various  training 
settings.  Groups  of  senior  technicians  in  the  specialty,  who  are 
thoroughly  familiar  with  the  work  of  the  specialty,  estimate  how  much 
training  time  is  currently  devoted  to  reach  minimum  required  proficiency 
for  the  various  groups  of  tasks  for  the  following  training  settings: 
classroom,  correspondence  courses  such  as  career  development  courses 
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(CDCs),  hands-on  training  (FTDs,  MTTs,  etc.),  supervised  hands-on 
experience  on  the  job  (OJT),  and  other  programs.  These  raters  are  also 
asked  to  provide  training  time  estimates  for  "ideal"  training  (i.e.,  the 
most  effective  mix  of  types  of  training).  Finally,  the  raters  are  asked 
how  long  it  would  take  in  each  setting  to  train  the  TTM,  if  the  training 
was  provided  in  only  that  setting  (e.g.,  only  classroom  training,  only 
correspondence  course  training).  This  "maximum  effective  training" 
may  not  always  yield  full  proficiency;  it  may  not  be  possible  to  fully 
train  a  TTM  in  one  or  more  of  the  settings  alone.  In  these  cases,  the 
respondents  indicated  the  proficiency  level  reached  in  each  setting  as  a 
percent  of  full  proficiency.  Thus,  each  SME  provided  six  allocation 
judgments  for  each  TTM:  four  that  related  the  maximum  training  time  in 
each  of  four  settings  to  the  proficiency  level  reached,  one  for  the 
current  allocation,  and  one  for  the  most  preferred  allocation  of  training. 

Difficult  issues  such  as  proficiency  measurement  and  the  description 
of  partial  allocations  of  training  to  different  training  settings  were 
generally  resolved  to  the  satisfaction  of  the  many  SMEs  who  were  involved 
in  the  development  of  these  procedures. 

What  Is  Proficiency?  For  TDS,  proficiency  was  defined  as  a  percentage 
of  the  training  needed  by  an  average  individual  to  reach  the  minimum 
required  standard  for  each  TTM  (the  "go/no  go"  level  of  OJT  =  100% 
proficiency).  SMEs  generally  understood  and  were  able  to  estimate  degrees 
of  proficiency  expressed  in  this  way.  SMEs  are  also  able  to  reliably  and 
consistently  describe  the  current  training  pattern  in  terms  of  how 
training  in  each  type  of  setting  contributes  incrementally  to  the 
attainment  of  full  proficiency  (i.e.,  the  partial  proficiency  achieved). 
They  also  easily  conceptualized  other  combinations  of  training  for 
reaching  the  same  goal  (i.e.,  alternative  training  allocations). 

Allocation  Functions?  It  was  hypothesized  that  proficiency  gain  from 
training  in  a  setting  would  be  greatest  initially  and  would  decline  as 
more  training  was  provided  in  that  setting.  Eventually,  there  would  be  no 
more  gain  from  providing  training.  Thus,  the  predicted  relationship 
between  proficiency  and  time  in  a  training  setting  is  that  of  initial  gain 
followed  by  proficiency  leveling-off,  a  negatively  accelerated  curve. 
This  general  set  of  relationships  is  depicted  in  Figure  2;  these  curves 
are  theoretical  estimates  which  illustrate  the  expected  form  of  the 
relationship. 

This  relationship  can  also  be  expressed  in  the  following  polynomial 
regression  equation: 

Proficiency  =  a  *  class-hours  -  b  *  class-hours**2  + 
c  *  self-study-hours  -  d  *  self-study-hours**2  + 
e  *  field-training-hours  -  f  *  field-training-hours**2  + 
g  *  work-hours  -  h  *  work-hours**2, 

where  "a"  through  "h"  are  coefficients  to  be  estimated  by  multiple  regres¬ 
sion,  **2  indicates  squaring,  and  the  regression  equation  is  constrained  to 
pass  through  the  origin  (there  is  no  constant  for  the  Y  intercept). 
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Figure  2.  Hypothesized  Relationship  Between  Hours  of  Training  in  a 
Setting  and  Proficiency  Gain  (from  Perrin  et  al.,  1988,  p.  29). 


This  model  involves  specific  hypotheses  about  the  nature  of  the 
relationship  between  setting  training  hours  and  proficiency.  Specifical¬ 
ly,  controlling  for  training  in  each  of  the  other  training  settings,  the 
first-order  parameter  is  specified  to  be  positive  and  the  second-order 
parameter  is  negative,  yielding  the  predicted  negatively  accelerated 
curve.  Across  the  four  AFSs  studied  during  TDS  development,  this 
statistical  model  was  strongly  supported.  Statistical  estimates  consis¬ 
tent  with  the  polynomial  regression  equation  were  found  in  well  over  90% 
of  the  allocation  curves  in  all  four  specialties  (Perrin  et  al.,  1988). 

There  are  two  additional  sources  of  support  for  the  conceptualization 
of  proficiency  gain  as  a  negatively  accelerated  function  of  training  time 
in  a  setting.  First,  the  overall  fit  of  the  polynomial  regression  model 
was  found  to  be  quite  good,  averaging  over  65%  (multiple  R  squared)  in  two 
specialties.  Second,  the  additional  variance  explained  by  the  second- 
order  terms  in  the  allocation  equations  was  substantial  (approximately  15% 
increase  in  R  squared  for  these  two  specialties) ,  indicating  that  simple 
linear  functions  are  not  sufficient  to  describe  proficiency  gains  from 
training  in  each  setting  (see  Figure  3  for  an  example  allocation  curve). 
A  curvilinear  model  is  much  more  descriptive  of  these  relationships 
(Perrin  et  al.,  1988). 
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Figure  3.  Example  Allocation  Curves  for  Aircraft  Environmental 
Systems  (AFS  423X1)  TTM  34,  Doppler  Sensor  Control  Boxes. 


Possible  alternative  allocations  range  (theoretically)  from  training 
everything  about  a  TTM  in  the  classroom  to  training  the  TTM  entirely  on  the 
job.  By  collecting  data  from  SMEs  on  a  few  possibilities  (i.e.,  current, 
"ideal,"  and  "maximum  effective"  training)  and  on  the  time  involved  for 
each,  allocation  curves  are  generated  which  represent  all  possible 
combinations.  Such  curves  permit  a  direct  translation  among  settings  of 
training  time  to  percent  of  required  proficiency  for  each  TTM,  thus 
facilitating  specification  and  evaluation  of  training  alternatives  (i.e., 
various  combinations  of  training  times  in  different  settings  that,  in 
total,  achieve  100%  of  the  required  proficiency). 

Allocation  curves  for  all  TTMs  of  a  specialty  derived  through  this 
survey  approach  give  the  TDS  maximum  flexibility  in  considering  different 
ways  of  dividing  training  among  training  settings,  as  well  as  identifying 
the  limits  of  each  type  of  training.  This  capability  serves  as  the 
foundation  for  developing  and  evaluating  alternative  patterns  of  training 
(as  will  be  discussed  in  the  Field  Utilization  Subsystem  section).  Thus, 
the  allocation  curves  are  a  very  significant  part  of  the  overall  TDS 
design;  their  development  and  validation  represent  a  substantial  advance  in 
training  decisions  technology. 
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2. 3. 1.3  Relationship  of  the  TCS  to  Other  Subsystems.  The  set  of  TTMs 
developed  and  validated  for  a  specialty  represents  one  of  the  major 
buildir''  blocks  for  the  TDS.  The  TTMs  are  a  major  input  to  the  FUS  and 
serve  as  descriptors  for  jobs  and  training  states  (courses,  OJT  programs, 
etc.).  Data  are  summarized  or  averaged  to  the  module  (TTM)  level,  such 
that  jobs  can  be  described  in  terms  of  percent  of  job  incumbents  performing 
each  TTM  and  the  total  time  spent  on  TTMs.  Training  course  content  is  also 
expressed  in  terms  of  hours  of  training  per  TTM.  Thus,  training  and  job 
content  share  a  common  set  of  terms.  The  TTMs  also  serve  as  a  foundation 
for  the  RCS  in  that  information  about  training  resources  required  and 
resource  availability  are  collected  on  a  TTM-by-TTM  basis.  This  greatly 
reduces  the  complexity  of  RCS  analyses. 

Training  allocation  curves  play  an  important  input  in  the  FUS  in  terms 
of  facilitating  development  and  evaluation  of  alternative  U  &  T  patterns. 
These  curves  make  possible  the  calculation  of  OJT  requirements  for  •  each 
U  &  T  pattern  simulation.  Allocation  curves  are  also  critical  to  the 
Resourc^/Cost  Subsystem  where  they  are  used  for  calculating  training  costs. 

2.3.2  Field  Utilization  Subsystem  (FUS^ 

The  FUS  pv'ovides  information  for  defining  training  and  job  assignment 
patterns  with  associated  management  preference  values  for  both  current  and 
several  plausible  alternative  approaches  to  training,  assigning,  and 
utilizing  airmen  in  a  particular  specialty  over  the  span  of  their  Air  Force 
careers  (see  Yadrick,  Knight,  Mitchell,  Vaughan,  &  Perrin,  1987).  Both 
training  content  and  job  descriptions  are  represented  as  collections  or 
sets  of  TTMs. 

2. 3. 2.1  Current  Utilization  and  Training  CU  &  T)  Pattern  Component.  A 
description  of  the  current  U  &  T  pattern  for  a  given  specialty  is  a  necessary 
starting  point  both  for  understanding  the  current  situation  and  for 
developing  possible  management  choices.  Appropriate  data  must  be 
synthesized  from  a  variety  of  sources — most  notably  from  occupational 
survey  (OS)  data,  the  Uniform  Airman  Record  (UAR),  the  Pipeline  Management 
System  (PMS),  AFR  50-5,  TDS  surveys,  and  informal  interaction  with 
functional  managers,  training  managers,  and  field  representatives — to  gain 
a  complete  picture  of  the  present  AFS  training  and  personnel  assignment 
flows.  Such  information  must  be  summarized  and  displayed  to  Air  Force 
managers  in  a  simple,  effective  format  that  provides  a  brief  yet 
comprehensive  picture  of  the  specialty. 

A  key  element  in  defining  a  career  pattern  is  the  identification  of  the 
jobs  within  a  specialty.  A  job  is  a  group  of  related  positions  in  which 
many  of  the  same  tasks  are  performed;  a  position  is  a  unique  set  of  tasks 
performed  by  one  person  (Shartle,  1959;  see  also  AFR  35-2,  para.  1).  Each 
AFS  includes  a  number  of  jobs  that  vary  in  content  (tasks  performed) 
according  to  the  organizational  level,  unit  mission,  equipment  operated  or 
maintained,  level  of  experience  of  personnel,  and  a  number  of  other 
interrelated  factors  (Driskill  &  Mitchell,  1979). 

In  previous  Air  Force  R&D ,  methods  were  devised  for  the  analysis  of  OS 
data  collected  from  job  incumbents  to  identify  the  major  types  of  jobs 
which  exist  in  an  occupational  area  (Archer,  1966;  Christal,  1974;  Driskill 
&  Mitchell,  1979;  Morsh,  1964;  Ward,  1963).  Such  information  is  used  to 
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evaluate  the  way  in  which  the  specialty  is  organized  and  the  appropriate¬ 
ness  of  providing  initial  skills  training  for  various  specialty  tasks  (AFR 
35-2;  ATCR  52-22;  see  also  Ruck  et  al.,  1978,  1987).  In  addition,  such 
data  have  been  experimentally  processed  for  identifying  tasks  for  OJT 
(Datko,  Cassidy,  &  Ruck,  1982),  for  defining  safety  priorities  (Thompson  & 
Ruck,  1984),  and  for  other  advanced  uses. 

For  the  TDS,  job  types  were  identified  using  standard  occupational 
analysis  (OA)  methods.  Using  two  test  specialties,  several  attempts  were 
made  to  create  job  clusters  based  on  type  of  weapon  system  or  equipment 
maintained,  on  average  grade  of  personnel  performing  tasks,  and  on  other 
potentially  relevant  factors  for  two  test  specialties.  Such  statistical 
weighting  procedures  did  not  significantly  change  the  jobs  identified  in 
the  original  OA  job  typing  (Yadrick  et  al.,  1987).  Cross-KPATH  analysis 
was  attempted  with  three  specialties  where  more  than  one  occupational 
survey  report  (OSR)  was  available,  without  significant  results.  In 
addition,  first  enlistment  jobs  (1  to  48  months  Total  Active  Federal 
Military  Service  or  TAFMS)  were  contrasted  with  career  (49+  TAFMS)  jobs; 
however,  no  major  differences,  other  than  the  expected  amount  of 
supervisory-type  work,  were  identified.  Thus,  the  OA  job  types  appear  to 
be  a  realistic  foundation  for  describing  the  current  U  &  T  pattern  for  Air 
Force  enlisted  specialties  (Yadrick  et  al.,  1987).  [NOTE:  Advanced  job 
typing  procedures  are  currently  being  developed  and  tested  (Phalen,  Staley, 
&  Mitchell,  1987)  in  an  effort  to  make  the  process  more  efficient.] 

For  TDS,  OSR  data  are  reprocessed  to  create  more  concise  job 
descriptions  based  on  the  TTMs  of  the  specialty  from  the  TCS.  The  CO  DAP 
set  of  MODULE  programs  (Module  Title  File,  MODSET,  PRTMOD)  is  used  to 
create  a  new  data  base.  Four  indices  are  used  to  characterize  the  Job-TTM 
association: 

1.  the  sum,  across  people  and  tasks,  of  the  percent  time  spent 

performing  the  tasks  of  a  TTM; 

2.  a  running  cumulation  of  this  summed  percent  time  spent  index; 

3.  the  average  percent  time  spent  per  task  in  the  TTM;  and 

4.  the  average  percent  members  performing  across  TTM  tasks. 

The  cumulative  sum  of  percent  time  spent  per  module  (TTM)  was  the  only 
index  not  already  computed  in  ASCII  CO  DAP.  Through  coordination  with 
AFHRL's  Manpower  and  Personnel  Division,  this  function  was  added  to 
ASCII-CODAP  MODULE  programs,  thus  avoiding  any  need  to  write  TDS -specific 
software  for  this  part  of  the  FUS  (Yadrick  et  al./  1987). 

The  identification  of  all  training  courses  and  job  assignment  flows 
within  a  specialty  proved  to  be  a  greater  problem.  For  example,  not  all 
courses  (i.e.,  FTD  and  MTT  courses)  are  identified  in  the  typical  OSR. 
AFR  50-5  lists  specialty- specific  and  aircraft-specific  courses  but  does 
not  provide  detailed  information  on  course  content.  Personnel  data  files 
(UAR)  contain  very  limited  training  information  but  do  indicate  PME 
courses  attended.  They  also  give  assignment  histories,  but  these  often 
use  the  specialty  title  or  generic  skill-level  names  rather  than  job 
titles  equivalent  to  OSR  jobs.  The  PMS  contains  training  attendance  and 
completion  records  for  formal  courses  by  individual,  but  not  by 
specialty.  No  existing  source  could  provide  all  of  the  data  needed. 
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For  TDS  purposes,  such  information  was  synthesized  from  various  sources 
to  identify  specialty  courses,  relevant  generic  FTDs,  PME  programs,  and 
specialized  training  mechanisms,  such  as  the  Educational  Subject  Block 
Indices  (ESBIs)  used  in  Security  Police.  A  Job  and  Training  History  survey 
was  sent  to  a  representative  sample  of  experienced  job  incumbents  (and  a 
random  sample  of  first-job  personnel).  Respondents  identified  their 
present  and  previous  jobs,  and  listed  dates  of  attendance  for  all  training 
programs;  they  could  also  write  in  additional  training  programs.  These 
data  were  sorted  and  processed  to  estimate  rates  of  attendance  for  the 
relevant  courses  for  each  job,  attrition  rates,  and  assignment  flows 
(average  length  of  assignment,  PME  course  attendance  points,  etc.).  Such 
information  was  compared  to  the  OSR  or  data  from  other  objective  sources. 

Graphic  flow  patterns  were  developed  (see  example  in  Figure  4),  as  well 
as  concise  summaries  of  the  data.  Such  displays  were  validated  with 
specialty  SMEs  in  subsequent  interviews  or  meetings  whenever  possible.  The 
graphic  display  provides  a  sense  of  the  flow  of  individuals  through 
training  programs  and  jobs  but  does  not  lend  itself  to  summarizing  the 
various  types  of  quantitative  information  involved,  such  as  the  number  of 
individuals  entering  the  field  each  year,  the  various  probabilities  of 
reassignment  among  jobs,  attending  advanced  technical  training,  or 
participating  in  PME  courses.  Such  data,  which  are  critical  aspects  of 
describing  the  specialty,  are  better  conceptualized  as  a  series  of 
data  matrices. 

FIRST  SECOND  THRO  FOURTH  FIFTH 

JOB  JOB  008  JOS  JOB 


AIRCRAFT  FTOS  PROBABLE  POSSIBLE  NCO  SR  NCO 

EQUIPMENT  FTOS  FTOS  FTOS  ACADEMY  ACADEMY 

GENERC  FTDS  PAC I  LEADERSHIP 

PV€I  SCHOOL 

SLEVELCCC 


Current  Utilization  and  Training  Pattern  for  Avionic 


Inertial  and  Radar  Navigation  Systems  (AFS  328X4). 
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Such  data  matrices  summarize  the  probabilities  of  individuals  being 
assigned  to  various  jobs,  of  attending  specialty- specific  and  PME  training 
programs,  and  of  exiting  jobs  for  reassignment  or  to  leave  the  specialty 
(or  the  Air  Force).  To  adequately  model  the  complex  flows  of  personnel 
through  specialty  jobs  and  training  programs,  the  TDS  employs  a  dynamic 
simulation  approach,  which  gives  the  analyst  or  researcher  the  capability 
to  change  specified  values  (i.e.,  number  entering,  relative  assignment 
probabilities,  attrition  rates,  etc.).  Thus,  the  current  U  &  T  pattern  can 
be  modified  to  consider  possible  changes  of  the  present  approach  to 
providing  training  or  assigning  personnel  within  the  specialty;  any  such 
modification  is  considered  an  alternative  U  &  T  Pattern. 

2. 3. 2. 2  Alternative  U  &  T  Pattern  Component.  The  second  FUS  component 
develops  alternative  U  &  T  patterns  which  managers  and  commanders  might  wish 
to  consider  and  evaluate.  Such  alternatives  could  include  patterns  that 
minimize  initial  skills  training  (limit  the  number  of  first-term  jobs), 
that  represent  present  proposals  for  change  of  the  specialty  (e.g.,  the 
programmed  expansion  of  Air  Base  Ground  Defense  training  to  all  new 
Security  and  Law  Enforcement  personnel),  that  represent  logical 
restructuring  of  work  in  a  specialty,  that  involve  reorganization  of 
training  for  a  specialty,  or  that  might  possibly  result  from  expected 
procurement  of  new  equipment  or  new  procedures.  TDS  handles  most  proposed 
changes  as  alternative  U  &  T  patterns  and  builds  displays  and  data  files  to 
simulate  the  consequences  of  these  changes.  Figure  5  displays  an  example 
of  an  alternative  U  &  T  pattern  for  the  current  U  &  T  pattern  shown  in  Figure 
4.  Note  in  this  example,  the  major  change  is  to  eliminate  FTD  training  by 
adding  trailer  courses  immediately  after  resident  training  for  small 
(Fighters/Recon)  and  large  (Strategic/Airlift)  aircraft  systems.  The 
objective  in  this  alternative  is  to  ensure  aircraft  systems-specific 
training  for  everyone  (FTD  attendance  is  typically  very  limited)  to  ensure 
job  qualification. 

For  the  TDS  development  effort,  ideas  for  alternative  patterns  were 
developed  concurrently  with  the  creation  of  the  current  U  &  T  model. 
Interviews  with  HQ  USAF,  major  command  (MAJCOM),  and  functional  managers 
were  supplemented  with  information  from  technical  school  instructors  and 
other  SMEs.  Other  alternatives  were  constructed  based  on  rational  job 
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NO  FTD 

Figure  5.  Example  of  an  Alternative  Utilization  and  Training  Pattern; 
Avionic  Inertial  and  Radar  Navigation  Systems  (AFS  328X4). 
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engineering  or  to  meet  some  objective  function  (i.e.,  reduce  the  number  of 
initial  job  possibilities,  minimize  initial  skills  training,  etc.). 
Whenever  possible,  developed  alternatives  were  validated  with  SMEs,  who 
examined  them  for  plausibility  and  potential  value.  In  most  cases,  the 
potential  alternative  U  &  T  model  was  created  by  modifying  some  major  element 
cf  the  current  U  &  T  model;  for  example,  the  simplest  change  would  be  to 
increase  or  decrease  the  flow  of  personnel  into  the  specialty. 

In  all  cases,  the  implications  of  change  had  to  be  evaluated  in  terms 
of  impact  on  flow  of  assignments,  school  attendance  rates,  etc.  The 
structure  of  the  FUS  specialty  model  facilitates  this  process  since  it  is 
necessary  to  consider  how  any  change  will  be  implemented  as  changes  in  the 
data.  [In  extreme  cases,  such  as  where  new  equipment  is  procured  or  where 
entirely  new  jobs  are  created,  new  data  files  would  need  to  be  constructed 
including  new  job  descriptions  and  new  training  programs,  as  well  as  making 
modifications  to  job  and  training  assignment  flows.] 

2. 3. 2. 3  FUS  Flow  Simulation  Program:  A  Dynamic  Modeling  Approach.  A 
computer-based  simulation  program  was  developed  for  TDS  as  a  tool  to 
facilitate  analysis  of  current  and  alternative  U  &  T  patterns,  and  to  permit 
calculation  of  total  specialty  training  requirements  for  each  pattern. 
Such  a  simulation  program  is  absolutely  necessary  for  the  TDS  if  we  are  to 
model  and  evaluate  changes  to  any  of  the  variables  used  to  characterize 
specialty  jobs,  job  content,  training  programs,  course  content,  assignment 
probabilities,  training  capacities  of  units,  and  training  costs  or 
resources. 

The  dynamic  simulation  program  is  a  major  innovation  in  technology  in 
terms  of  better  estimation  of  OJT  requirements  for  all  jobs  in  the 
specialty.  It  considerably  extends  earlier  research  aimed  at  adapting 
econometric  and  manpower  modeling  to  the  issues  of  personnel  assignment 
flows  and  training  costs  in  operational  units  (Eisele,  Bell,  &  Laidlaw, 
1978;  Rueter,  Bell,  &  Malloy,  1980).  Such  a  simulation  is  critical  for 
calculating  OJT  requirements  and  costs  realistically;  any  changes  in  job 
content,  training  programs,  or  even  assignment  probabilities  have  an  impact 
either  directly  or  indirectly  on  the  requirement  for  OJT. 

The  program  is  a  dynamic  simulation  system  which  processes  descriptive 
data  files  in  such  a  way  as  to  show  total  numbers  of  personnel  flowing 
through  jobs  and  training  programs  over  an  extended  period  of  time.  Data 
can  be  examined  by  any  specified  period  of  time;  training  flows  are 
generally  expressed  as  annual  rates  where  assignment  probabilities  are  more 
realistically  portrayed  for  2-  or  3-year  intervals  equivalent  to  the 
typical  Air  Force  specialty  job  assignment.  Such  dynamic  processing  of 
several  models  (U  &  T  patterns)  of  a  specialty  provides  the  needed 
flexibility  to  project  future  requirements  and  assess  the  consequences  of 
proposed  changes. 

Existing  programs,  such  as  the  Simulation  Language  for  Alternative 
Modeling  (SLAM),  were  found  to  be  not  fully  adequate  for  handling  the 
complex  specialty  models  needed  in  the  TDS.  Several  systems  were  examined 
and  tested  but  were  found  to  be  unable  to  handle  the  complexities  of  most 
Air  Force  specialties  (Yadrick  et  al.,  1987).  Thus,  new  simulation 
software  was  designed  and  tested  which  would  meet  the  specific  requirements 
of  the  TDS. 
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The  new  program  is  titled  UTPSIM.  the  U  &  T  Pattern  Simulation  Program. 
It  uses  data  from  the  10  files  listed  below  to  generate  a  flow  pattern  of 
entities  (hypothetical  individuals)  moving  from  initial  training  course  (s) 
through  first  jobs  (and  associated  training)  to  advanced  courses  or  new 
jobs  over  a  full  career.  It  also  accounts  for  career  field  attrition  (both 
cross-training  to  other  specialties  and  leaving  the  Air  Force),  FTD  and  PME 
requirements,  and  other  specialty-specific  factors.  Files  used  in 
generating  UTPSIM  models  include: 


RUNPAR 

JOBIDS 

JOBTTM 

TRNPAR 

JOBTRN 

TIMTRN 

TJBTRN 

ENTRYS 

FRM JOB 

TOJOBS 


Run  parameter  file  (includes  required  specifications) 
Titles  and  job  identification  information 
Percent  people  performing  TTMs  per  job 
Titles,  type,  and  length  of  courses 
Job-driven  training  probabilities  (Option  1) 
TAFMS-driven  training  probabilities  *  (Option  2) 
Job-  and  time-driven  training  probabilities  (Option  3) 
Number  of  people  entering/transition  probabilities 
Probabilities  of  entities  exiting  Specialty  jobs 
Probabilities  of  entities  entering  Specialty  jobs 


*  TAFMS  =  Total  Active  Federal  Military  Service 

The  output  of  the  UTPSIM  program  is  an  Output  Entity  History  File 
(OUTEHF)  which  contains  the  job  and  formal  training  history  of  each 
individual  (entity).  For  any  given  set  of  input  data  (current  or 
alternative  U  &  T  patterns,  or  variations  of  input  parameters,  such  as 
annual  input),  the  dynamic  simulation  calculates  how  individual  entities 
enter  the  system,  move  through  training  and  job  states,  and  exit  the 
specialty  (or  the  Air  Force).  These  data  are  processed  in  two  additional 
programs  (HIS  CRN  and  TRNPRF)  to  produce  the  final  FUS  output,  as  shown  in 
Figure  6. 

The  History  Screening  ( HIS  CRN)  program  is  needed  to  eliminate  those 
entities  who  are  not  in  the  equilibrium  window  (the  period  of  interest  in 
the  simulation)  and  thus  are  not  relevant  for  determining  total  OJT 
requirements  of  the  specialty.  The  Training  Proficiency  (TRNPRF)  program 
computes  the  total  amount  of  OJT  needed  in  the  specialty  for  all 
individuals  to  achieve  required  proficiency  on  the  tasks  of  the  TTMs 
involved  in  their  jobs. 

The  TRNPRF  program  uses  two  new  data  files  not  previously  noted:  the 
Allocation  Curve  file  and  the  TIM -Course  file.  The  Allocation  Cup/e  file 
contains  parameters  derived  in  the  TCS  which  reflect  how  training  is  to  be 
allocated  to  various  training  settings.  There  is  a  separate  allocation 
curve  for  each  TTM  for  each  setting,  and  proficiency  is  assumed  to  be 
additive  across  settings.  There  are  many  different  combinations  of  how 
training  could  be  delivered  which  would  result  in  achievement  of  required 
proficiency  (100%  =  go  level  under  AF  OJT  go/no  go  concept)  on  TTM  tasks. 
Allocation  curves  also  set  an  upper  bound  on  the  number  of  hours  that  are 
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worth  providing  in  a  setting  in  terms  of  the  maximum  proficiency  that  can 
be  achieved  in  that  setting  (Perrin,  Knight,  Mitchell,  Vaughan,  &  Yadrick, 
1988;  Vaughan,  Mitchell,  Marshall,  Feldsott,  &  Rueter,  1988).  This  level 
sets  an  effective  limit  on  the  TIM  training  hours  which  should  be 
delivered  in  the  setting. 

The  Allocation  Curve  Parameter  file  contains  a  complex  array  of  data 
for  each  TTM-by-setting  combination  (see  Table  1).  Such  data  include: 
TIM  ID  Number,  Setting  ID  Number,  Beta  (squared  regression  coefficient  of 
the  allocation  curve  for  the  TTM-by-setting) ,  Alpha  (regression  constant 
for  the  particular  function) ,  Current  Allocation  Hours  (estimated  hours  on 
the  TIM  under  current  training) ,  Ideal  Allocation  Hours ,  Maximum  Time  (the 
maximum  effective  training  hours  for  the  TIM) ,  Minimum  Time  (minimum  hours 
to  achieve  full  proficiency) ,  Maximum  Proficiency  (the  maximum  obtainable 
in  this  setting  on  the  TIM) ,  and  Current  Allocation  Preference  (rating,  on 
a  0  to  4  scale) . 

All  these  types  of  data  are  necessary  for  the  TRNFRF  to  be  able  to 
calculate  COT  requirements  for  each  TIM.  For  details  of  how  the  TRNPRF 
program  operates  to  perform  these  calculations,  see  the  TPS  Procedural 
Guide  (Vaughan,  Mitchell,  Marshall,  Feldsott,  &  Rueter,  1988).  Fora 
discussion  of  how  allocation  curves  are  developed,  see  the  TCS 
Administrative  Report  (Perrin,  Knight,  Mitchell,  Vau^ian,  &  Yadrick, 
1988) . 

Table  1.  Example  Allocation  Curve  Parameter  File  Data  for  Four  TTMs 


TIM  Setting 

Current 

Ideal 

Max 

Min 

Max 

Current 

ID 

ID 

Beta 

Alpha 

alloc. 

hours 

time 

time 

pref 

1 

1 

0.0000 

0.1295 

19.55 

9.09 

298.38 

0.00 

38.6 

0.96 

1 

2 

0.0033 

0.9912 

18.10 

16.09 

48.04 

0.00 

40.0 

0.94 

1 

3 

0.1578 

6.3350 

2.26 

4.61 

11.78 

0.00 

52.7 

0.67 

1 

4 

0.0536 

4.6010 

18.62 

15.32 

25.64 

0.00 

82.7 

0.78 

2 

1 

0.2049 

7.1250 

6.15 

5.96 

10.85 

0.00 

53.2 

0.97 

2 

2 

0.0232 

2.0190 

3.24 

4.83 

24.92 

0.00 

35.9 

0.92 

2 

3 

0.2289 

7.1360 

3.44 

2.65 

15.59 

0.00 

55.6 

0.94 

2 

• 

4 

• 

0.1149 

• 

6.0240 

• 

6.84 

• 

7.40 

• 

26.21 

• 

0.00 

• 

79.0 

• 

0.97 

• 

• 

8 

• 

• 

1 

• 

• 

0.0000 

• 

• 

0.7900 

• 

• 

25.63 

• 

• 

20.71 

• 

• 

49.73 

• 

• 

0.00 

• 

• 

39.3 

0.83 

8 

2 

0.0059 

0.1043 

8.84 

8.84 

8.84 

0.00 

0.5 

1.00 

8 

3 

0.0154 

1.4820 

10.87 

14.43 

48.12 

0.00 

35.7 

0.89 

8 

• 

4 

• 

0.2138 

• 

9.4750 

• 

8.48 

8.48 

• 

8.48 

• 

2.77 

• 

65.0 

• 

1.00 

• 

• 

57 

• 

1 

• 

• 

1.2340 

• 

• 

16.8900 

0.90 

• 

• 

0.92 

• 

• 

3.92 

• 

• 

0.00 

• 

47.3 

• 

0.99 

57 

2 

11.3200 

34.9400 

0.69 

0.97 

1.54 

0.00 

27.0 

0.72 

57 

3 

6.6640 

41.3200 

0.41 

0.65 

3.10 

0.00 

64.1 

0.90 

57 

4 

2.3450 

32.7600 

1.80 

1.29 

3.67 

0.00 

88.6 

0.79 
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The  TIM-Course  file  is  based  on  data  collected  via  a  survey  of  the 
technical  training  center  (TTC)  and  FTD  instructors  as  well  as  COT 
trainers  in  representative  field  units.  Each  course  is  assessed  by  one  or 
more  qualified  SMEs  (typically  course  instructors  and/or  course 
supervisors)  and  is  characterized  in  terms  of  the  number  of  hours  of 
classroom  time,  hands-on  training,  and  self-study  time  spent  for  each 
relevant  TIM  (see  Table  2  for  an  example).  Additional  categories  (such 
as  administrative  time,  testing  time,  etc.)  are  allowed;  the  total  time 
should  equal  the  course  length  specified  in  AFR  50-5. 


Table  2.  Example  Course  Description  Using  TTMs  (AER32834  001  -  Avionic 
Inertial  and  Radar  Navigation  Systems);  30  wks,  4  days  (as  of  June  1986) 

Classroom  Hands-on  Self-study 


Task  Training  Modules 

hours 

hours 

hours 

Block  1  -  Electronics  Fundamentals 

672.0 

Block  2  -  Avionics/Navigational  Systems 

4 .  Training/CJT  Program 

6.0 

2.0 

9.  Inspect  Workcenter  Equipment 

3.0 

6.0 

2.0 

17.  Liaison  with  Job  Control 

2.0 

2.0 

1.0 

22.  Maintain/Prepare  Tech.  Orders 

1.0 

.5 

24.  Nav.  Equip.  Maint.  On  Acft 

20.0 

15.0 

8.0 

25.  Inertial  System 

75.0 

22.0 

12.0 

26.  Navigation  Unit  Components 

9.5 

1.0 

2.5 

31.  Doppler  Systems  Off-Equipment 

16.0 

4.0 

32.  Doppler  Systems  On-Equipment 

19.0 

16.0 

33.  Doppler  Nav.  Camp.  Cards 

.5 

34.  Doppler  Sensor  Control  Boxs 

.5 

1.0 

36.  A/C  Wiring  Harness 

2.0 

44.  Gen  Purpose  Nav  Coup  On  Acft 

32.0 

6.0 

10.0 

45.  Gen  Purpose  Nav  Comp  Off  Acft 

33.0 

33.0 

4.0 

50.  Relay  Panels  &  INS  Temp  Bulbs 

1.0 

.5 

59.  Maintain  WRCS 

26.0 

25.0 

6.0 

61.  Weapons  Release  System 

24.0 

12.0 

6.0 

67.  Heading  Computers/Recon.  Adapt. 

2.0 

2.0 

1.0 

68.  Maint  Inert.  Corrp/Off  Equipment 

31.0 

24.5 

8.0 

73.  Maint/Program  Nav  Units 

5.5 

3.5 

2.0 

Subtotals  (Block  2) 

309.0 

173.0 

65.5 

Block  1  &  2  Classroom  +  Hands-On 

1154 . 0 

Additional  Course  Hours: 

Misc.  (Career  progression,  etc.) 

20.0 

Administration  (Tests,  processing,  etc.)  32.0 

Military  training  (FT,  drill  &  ceremonies,  etc.)  26.0 


Overall  AER  Course  Hours  1232.0 
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Data  for  all  courses  of  the  specialty  are  combined  into  a  matrix  of 
courses  versus  TTMs;  cell  entries  represent  total  hours  of  instruction 
(classroom  plus  hands-on  training)  provided  by  each  course.  By  describing 
all  courses  in  terms  of  TTMs  (as  jobs  were  described  earlier),  it  becomes 
possible  to  vary  course  content  (presence  or  absence  of  TTMs)  as  well  as 
hours  of  training  per  TTM  in  calculations  of  total  specialty  training 
requirements  (in  the  TRNPRF).  This  provides  the  structured  flexibility 
needed  in  the  system  to  be  able  to  consider  alternatives  to  the  courses 
and  training  hours  of  the  current  utilization  and  training  pattern. 

2. 3. 2. 4  Management  Preferences  Component;  Comparing  Managers1 
Preferences.  A  third  component  of  the  TCS  subsystem  consists  of  a 
methodology  for  obtaining,  analyzing,  and  displaying  managers'  preferences 
among  various  possible  training  and  assignment  patterns.  Such  preference 
data  are  collected  from  mid-level  training  managers,  functional  managers 
(HQ  USAF  and  MAJCOM),  and  other  senior  AFS  personnel;  this  permits  a 
comparison  among  the  various  types  of  managers,  to  make  visible  any 
possible  policy  differences.  If  the  U  &  T  preferences  are  similar  for 
various  groups  of  raters,  they  can  simply  be  averaged;  if  they  represent 
distinctly  different  policies,  then  senior  Air  Force  managers  must  decide 
which  preferences  to  use  in  making  decisions  on  specialty  restructuring  or 
solving  optimization  questions  in  the  system. 

The  U  &  T  pattern  narratives  and  diagrams  are  assembled  into  survey 
booklets  and  administered  to  managers  of  the  specialty.  It  is  best  to  do 
this  in  a  structured  interview  or  supervised  group  administration 
session.  Meetings  of  a  TPT,  U  &  TW,  or  other  specialty  conference  are 
excellent  opportunities  for  such  data  collection.  Group  sessions  or 
structured  interviews  ensure  standardized  survey  administration  and  allow 
managers  to  ask  clarifying  questions.  Typically,  as  involved  AFS 
managers,  participants  have  their  own  ideas  for  changes  in  the  AFS 
structure  or  training  programs;  such  suggestions  need  to  be  documented  for 
further  evaluation.  Survey  booklets  should  include  an  open-ended  option 
where  managers  can  describe  and  rate  "Other"  alternatives. 

Results  of  preference  surveys  need  to  be  analyzed  in  terms  of  their 
internal  consistency  and  possible  group  differences.  An  analyst  should 
check  for  such  differences  through  subgroup  comparisons  of  rating 
patterns.  (GRPREL  programs  are  available  in  ASCII  CO  DAP  to  accomplish  this 
assessment.)  Logical  candidates  for  comparison  are  field  (MAJCOM)  versus 
training,  HQ  or  functional  versus  training,  etc.  If  no  major  group 
differences  in  rating  patterns  are  found,  then  analysis  can  proceed  to 
build  displays  of  preferences  among  the  alternative  U  &  T  patterns 
considered.  If  there  are  significant  differences  among  rating  patterns, 
then  more  complex  displays  will  be  needed  to  properly  characterize 
preferences.  Results  should  be  included  in  a  formal  report  or  presented 
to  specialty  conferences  (TPT,  U  &  TW,  etc.)  for  consideration.  An  example 
of  such  a  summary  of  U  &  T  preferences  is  displayed  in  Table  3. 

It  should  be  noted  that  in  this  hypothetical  example,  each  group  has 
its  own  preferred  alternative;  the  overall  average  is  heavily  influenced 
by  the  largest  group  in  the  sample  (MAJCOM  managers).  In  most  AFSs 
studied,  differences  were  not  this  clearcut.  The  point  of  the  example  is 
that  when  such  policy  differences  are  present,  they  need  to  be  made 
visible  as  a  starting  point  for  discussion  and  negotiation. 
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Table  3.  Group  Preferences  for  U  &  T  Pattern  Alternatives 


Type  Of 
Manaaer 

CURRENT 

U&T 

AIT.  1 

AIT.  2 

AIT.  3 

AIT.  4 

AIT.  5 

AIT.  i 

HQ  USAF 

(3) 

6.7 

7.0 

4.7 

5.7 

3.7 

5.7 

5.3 

MAJOQM 

(6) 

4.8 

4.5 

3.7 

3.2 

1.7 

3.5 

8.2 

TRAINERS 

(4) 

6.25 

4.0 

5.5 

8.0 

2.0 

7.0 

4.5 

PERSONNEL  (2) 

3.5 

2.0 

5.0 

4.5 

8.5 

7.0 

6.0 

OVERALL 

(15) 

5.39 

4.53 

4.55 

5.15 

3.09 

5.34 

6.34 

1  =  lew  preference,  9  =  high  preference 


In  the  TDS,  management  preferences  are  not  a  final  decision,  but, 
rather,  serve  as  the  starting  point  for  evaluating  possible  alternative 
decisions  on  the  basis  of  their  potential  desirability,  costs,  and 
consequences.  Managers  may  elect  to  have  only  selected  (highly  preferred) 
alternatives  evaluated  further,  or  they  may  decide  to  construct  some  new 
alternative  (a  composite  of  several  proposals  or  new  ideas)  which  may 
require  new  data  gathering  for  further  consideration  and  evaluation.  In 
any  event,  same  products  of  the  FUS  and  TCS  subsystems,  representing 
considered  managers'  decisions,  become  basic  input  variables  for  the 
operation  of  the  Resource/Cost  Subsystem. 

2.3.3  Resource/Cost  Subsystem  fRCS) 

The  RCS  was  developed  to  provide  TDS  three  distinct,  yet  interrelated, 
analytic  capabilities: 

1.  determination  of  the  types  and  amounts  of  resources  required  to 
provide  training  on  each  TIM  in  each  training  setting,  and 
estimation  of  the  amounts  of  those  resources  available  for  use  in 
providing  training  at  each  site; 

2.  assessment  of  the  capacities  of  various  sites  to  accommodate 
differing  volumes  of  training  on  different  combinations  of  TIMS  in 
different  training  states,  where  a  training  state  consists  of  a 
set  of  specific  amounts  of  training  conducted  on  specific  TIMs  in 
particular  training  settings;  and 

3 .  estimation  of  the  variable  costs  that  must  be  incurred  in 
providing  training  on  each  TIM  in  each  training  setting,  and  in 
providing  particular  volumes  of  training  in  specific  training 
states. 

To  accomplish  these  objectives,  the  RCS  is  structured  into  three 
analytic  components:  a  Resource  Requirements  Component  (RRC) ,  a  Training 
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Capacity  Component  (TCC),  and  a  Cost  Estimation  Component  (CEC).  These 
components  use  input  files  from  the  TCS  and  FUS;  compile  resource 
requirements,  availability  and  cost  factor  data;  and  interact  with  one 
another  to  generate  resource  and  cost  estimates  (see  Figure  7;  see  also 
Rueter,  Vaughan,  &  Feldsott,  1988).  Training  resource  requirements  are 
classified  into  several  categories  (such  as  variable  versus  fixed, 
exclusive  versus  shared)  to  facilitate  estimating  training  capacities  and 
training  costs  (see  Figure  8). 

2. 3. 3.1  Resource  Requirements  Component.  The  RRC  performs  five  data 
development  functions.  Specifically,  it  (a)  determines  the  specific  types 
of  resources  required  to  perform  training  on  each  TTM  in  each  training 
setting,  (b)  estimates  the  quantity  of  each  identified  type  of  resource 
required  for  training  each  TTM  in  each  setting,  (c)  produces  compilations 
of  those  estimates  classified  on  the  basis  of  the  ways  in  which  the 
corresponding  types  of  resources  affect  variable  training  costs  and 
training  capacities,  (d)  estimates  the  quantities  of  those  resources 
available  for  the  provision  of  training  at  various  actual  sites,  and  (e) 
delineates  an  appropriate  set  of  representative  sites  for  the  particular 
specialty. 

Inputs  to  this  component  include:  TTM  definitions  and  amounts  of  time 
allocated  for  training  the  various  TTMs  in  different  training  settings 
(from  the  TCS),  and  preliminary  lists  of  resources  required  for  training 
each  TTM  in  each  setting  (collected  via  a  Training  Requirements 
Questionnaire  administered  to  school  and  field  trainers;  see  example  in 
Figure  9 ) .  Based  on  these  inputs,  an  analyst  identifies  representative 
sites  and  develops  the  basic  data  for  use  in  estimating  training  capacities 
and  costs  within  the  other  two  RCS  components. 

Representative  sites  (TTC  courses,  FTD  courses,  operational  units)  are 
identified  to  account  for  important  locational  variations  in  travel  and 
temporary  duty  (TDY)  costs,  training  loads,  missions,  resource 
availability,  and  other  factors.  The  use  of  representative  sites  permits 
more  economical  collection  of  cost  data  and  simplifies  comparisons  of 
resource  availability  and  resource  requirements.  A  key  consideration  in 
determining  resource  availability  is  the  identification  of  the  minimum 
resources  required  for  operational  duties  at  representative  sites,  since 
operational  requirements  place  real  constraints  on  the  training  capability 
of  a  unit. 

2. 3. 3. 2  Training  Capacity  Component.  The  TCC  evaluates  the  capacities 
of  various  representative  sites  to  provide  training  in  appropriate  settings 
on  different  combinations  of  TTMs  and  in  training  volumes  that  are 
compatible  with  the  U  &  T  patterns  identified  in  the  FUS.  Inputs  to  this 
component  consist  of  the  following:  TTM  combinations  and  training  volumes 
for  the  various  U  &  T  patterns  (from  the  FUS),  predicted  amounts  of  specific 
resources  required  for  the  provision  of  training  on  each  TTM  in  each 
training  setting  (in  the  form  of  regression  equations  from  the  RRC),  and 
availabilities  of  those  resources  for  providing  training  at  each  represen¬ 
tative  site.  Resource  availability  data  are  collected  in  a  Training 
Resources  Availability  survey  of  TTCs,  FTDs,  and  representative  field  units 
(see  example  in  Figure  10).  For  dedicated  training  resources,  data  on  the 
sharing  of  equipment  and  other  resources  must  be  collected  since  sharing 
has  a  potential  impact  on  training  capability,  and  may  vary  by  site. 
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^''\^RES0URCE  TYPE 
RESOURCE  USE^\^ 

VARIABLE:  PRIMARILY 
AFFECTING  VARIABLE 
TRAINING  COSTS 

FIXED: 

AFFECTING  TRAINING 
CAPACITY 

EXCLUSIVE  WITHIN  TTM 

RESIDENT  SCHOOL  STUDENT  TIME 
RESIDENT  SCHOOL  TRAINER  TIME 

MEDIA  AIDS 

SHARED: 

•  WITH  TRAINING  ON 
OTHER  TTMs  IN 

SAME  AFS 

COURSE  MATERIALS 

TRAINING  SUPERVISOR  TIME 
RESIDENT  SCHOOL  SUPPORT 
.  PERSONNEL  TIME 

TRAINING  EQUIPMENT 
LABORATORY  SPACE 

RESIDENT  SCHOOL  SUPPORT 
FACILITIES 

•  WITH  TRAINING 

ON  OTHER  AFSs 

TRAINING  BASE  SUPPORT 
PERSONNEL  TIME 

CLASSROOM  SPACE 

TRAINING  BASE  SUPPORT 
FACILITIES 

•  BETWEEN  TRAINING 
AND  OPERATIONAL 
DUTIES 

OJT  STUDENT  TIME 

OJT  TRAINER  TIME 

OPERATIONAL  EQUIPMENT 
OPERATIONAL  BASE 

SUPPORT  FACILITIES 

Figure  8.  The  RCS  General  Resource  Classification  Scheme 
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An  analyst  develops  estimates  of  the  capacity  of  each  representative 
site  to  accommodate  various  combinations  of  TTMs  and  training  loads,  and 
identifies  any  resource  limitations  that  constrain  representative  sites 
from  accommodating  particular  U  &  T  patterns.  When  such  constraints  are 
encountered,  they  are  displayed  in  the  COT  Capacity  Report  for  each  site  as 
"Trainees  Unsupportable"  (see  Table  4;  note  training  deficit  of  18 
trainees  imposed  by  Resource  39) . 


Table  4.  Example  of  a  Representative  Site  Training  Capacity  Report 
(Adapted  from  AFS  328X4  Report,  15  Feb  88,  Table  8.1) 

REPRESENTATIVE  SHE:  1 

Training  Capacity:  -  Upper  Bound:  18;  lower  Bound:  12 
Total  Trainees  Required:  30 


Resource 

Amount 

Amount 

Avail/Req 

Max  Tmees 

Trainees 

Trainees 

ID 

Avail . 

Required 

Ratio 

18 

5840.0 

0.2 

30917.65 

826695. 

26 

0 

27 

1800.0 

7.9 

227.27 

6782. 

29 

0 

39 

208.0 

511.9 

0.41 

12. 

30 

18  «« 

68 

1560.0 

2.0 

5902.30 

111941. 

30 

0 

91 

520.0 

1.9 

277.42 

7543. 

27 

0 

104 

5840.0 

29.8 

195.97 

5864. 

30 

0 

150 

17520.0 

61.6 

284.49 

8658. 

30 

0 

160 

52.0 

36.7 

1.42 

39. 

27 

0 

183 

8760.0 

0.7 

13431.01 

82007. 

6 

0 

Inherent  in  this  process  is  evaluation  of  the  feasibility  of  resource 
substitution,  the  impact  of  training  load  on  training  quality,  and  the 
possible  impact  of  training  on  mission  performance  (where  a  training 
deficit  exists,  then  resources  must  be  diverted  from  mission  performance 
to  provide  training,  or  the  unit  runs  the  risk  of  error  or  accident  which 
may  result  from  the  lack  of  complete  training) .  Training  capacity 
evaluations  use  statistical  training  resource  requirement  functions  and 
mathematical  programming  formulations  to  assess  various  training  options. 

2. 3. 3. 3  Cost  Estimation  Component.  The  CEC  computes  total  annual 
variable  costs  for  providing  training  of  each  TIM  in  each  training  setting 
(assuming  all  required  resources  are  available  in  sufficient  quantities) , 
and  then  compiles  the  cost  estimates  in  a  form  compatible  with  the 
estimates  developed  for  training  capacity.  Inputs  to  this  process 
include:  estimated  training  resource  requirements  (from  the  RRC) ,  training 
states  (i.e. ,  amounts  of  time  allocated  to  specific  TTMs  in  specific 
settings)  and  associated  training  volumes  compatible  with  various  U&T 
patterns  (from  the  FUS) ,  and  unit  resource  cost  factors  from  external  Air 
Force  data  sources  (TDY  costs,  instructor  salary  levels,  costs  of  training 
equipment  and  supplies,  etc.) .  By  applying  the  unit  cost  factors  to  the 
estimated  training  resource  requirements  for  the  specified  training  states 
and  training  volumes,  this  component  estimates  the  variable  costs  of 
conducting  training  in  each  training  setting  and  for  each  specified 
training  volume  in  the  corresponding  training  state. 
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Once  these  very  complex  basic  data  sets  have  been  developed,  they  must 
be  synthesized  and  processed  as  formatted  reports  in  such  a  way  as  to  be 
useful  for  Air  Force  decision  makers.  This  is  done  by  operation  of  the 
RCS  components  and  data  files  as  shown  in  Figure  11.  The  common  starting 
point  is  the  FUS  Output  File  (from  TRNPRF).  Multiple  processing  is 
performed  to  generate  training  hours  for  trainees,  trainers,  and  other 
resources,  for  both  classroom  and  OJT  requirements.  It  should  be  noted 
that  the  capacity  and  costing  programs  of  the  RCS  operate  in  parallel,  use 
some  common  data  files  as  input,  and  use  some  unique  files  as  well.  The 
major  products  of  the  RCS  are  data  files  and  reports;  these  are  organized 
by  representative  site,  training  setting,  and  job.  An  example  capacity 
report  was  given  earlier  (Table  4);  Table  5  provides  an  example  of  a 
cost  report  for  representative  sites  for  the  same  Air  Force  specialty. 

Such  unit-level  data  are  also  aggregated  to  derive  estimates  for  the 
total  AFS,  for  MAJCOMs,  and  for  individual  bases  in  a  summary  section  of 
the  report.  Separate  reports  of  training  capacity  and  costs  are  generated 
for  the  current  U  &  T  pattern  and  each  alternative  U  &  T  pattern  to  create 
multiple  RCS  output  files.  [For  more  details  on  the  RCS  and  its 
components,  see  Rueter  et  al.,  1988.]  RCS  data  files  serve  as  the  basis 
for  comparing  the  impact  of  various  suggested  AFS  changes,  and  for 
generating  reports  to  respond  to  managers'  inquiries,  through  the 
operations  of  the  Integration  and  Optimization  Subsystem  (IOS). 

2.3.4  Integration  and  Optimization  Subsystem  (IPS) 

The  IOS  ties  together  the  other  three  subsystems  into  one  overall 
functioned  system  (as  shown  earlier  in  Figure  1).  The  IOS  contains 
mechanisms  which  enable  system  modeling  and  optimization.  The 
interconnections  of  the  subsystems  provide  the  capacity  to  optimize 
measures  derived  from  one  subsystem  relative  to  constraints  obtained  from 
the  others,  and  to  simultaneously  process  data  files  derived  from 
different  subsystems.  The  IOS  also  provides  the  interface  with  users; 
that  is,  the  subsystem  receives  all  requests,  calls  appropriate  data  from 
the  other  subsystems  or  TDS  files,  and  creates  products  to  meet  the  users' 
needs.  In  all  of  its  functions,  the  IOS  governs  the  interaction  among  the 
TDS  subsystems  and  various  data  sources,  and  the  relationship  of  the 
system  with  various  types  of  users.  The  IOS  is  structured  to  perform 
three  types  of  functions; 

2,3.4. 1  Modeling  Functions.  The  IOS  processes  information  and  data 
files  from  the  TCS,  FUS,  and  RCS  to  create  various  models  of  the  AFS  under 
consideration.  The  basic  model  for  the  AFS  is  the  current  U  &  T  pattern; 
each  alternative  U  &  T  pattern  represents  some  change  to  the  current  U  &  T 
model.  This  approach  provides  maximum  flexibility  in  the  TDS,  since  an 
almost  infinite  number  of  possibilities  can  be  considered. 

The  modeling  functions  of  the  IOS  are  not  limited  to  examining  the 
alternative  U  &  T  patterns  formulated  in  the  FUS  (although  AFS  managers' 
preferences  are  collected  only  for  these  alternatives) .  Rather,  IOS 
modeling  provides  the  capability  to  change  any  input  variable  for  any 
program,  thus  permitting  examination  of  the  impact  of  such  change  on  the 
total  system.  For  example,  the  simplest  change  would  be  to  use  the 
current  FUS  model  and  raise  or  lower  the  number  of  personnel  entering  the 
ABR  course  (modify  the  Trained  Personnel  Requirement  or  TPR).  The  system 
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(see  Figure  6) 


RCS  Run  Parameters 


RRC  File 

Data  on  Site  -  Job 
Combinations 


Regression  Predictions 
(Hours 

TTM  -  Setting  —  Resource 
Combinations) 


1  SixCALCULATE  HOURS  Programs  j 

for 

Classroom  Trainees 

OJT  Trainees 

Classroom  Trainers 

OJT  Trainers 

Classroom  Resources 

OJT  Resources 

Resource 
Availability 
by  Site  Data 


COMPUTE 

CAPACITY 

PROGRAM 


COMPUTE 

COSTING 

PROGRAM 


Site  Location  Data 


Capacity  Report  |  |  Training  Cost  Report 


Figure  11.  Interrelationships  Among  RCS  Inputs,  Components,  and  Products 
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Table  5.  Total  Air  Force  Direct  OJT  Costs  by  Job  and  Representative  Site 
(Total  Trainee  and  Trainer  Salaries  +  Other  Costs) 

DIRECT  OJT  COSTS 

AVTONIC  INERTIAL  &  RADAR  NAVIGATION  SYSTEMS  (AFS  328X4) 

(in  Thousands  of  $) 


—  —  —  “ 

“  —  —  — 

R  E  P  R 

E  S  E  N 

TATI 

V  E 

SITE 

S - 

-  -  “  - 

JOB 

Site  1 

Site  2 

Site  3 

Site  4 

Site  5 

Site  6 

Site  7 

Site  8 

Site  9 

1 

0.0 

0.0 

0.0 

190.9 

95.5 

0.0 

916.5 

610.9 

0.0 

2 

1654.0 

6616.9 

0.0 

1985.1 

661.7 

0.0 

19339.6 

661.7 

0.0 

3 

3460.8 

0.0 

0.0 

598.5 

0.0 

0.0 

3877.1 

0.0 

0.0 

4 

279.7 

0.0 

0.0 

0.0 

0.0 

0.0 

69.9 

0.0 

0.0 

5 

358.5 

0.0 

0.0 

0.0 

167.3 

0.0 

740.9 

0.0 

0.0 

6 

577.2 

0.0 

0.0 

1487.4 

0.0 

0.0 

88.8 

777.0 

0.0 

7 

92.0 

57.5 

0.0 

218.5 

0.0 

0.0 

0.0 

0.0 

0.0 

8 

• 

93.5 

• 

2032.9 

• 

0.0 

• 

4392.9 

• 

0.0 

• 

0.0 

• 

0.0 

• 

397.2 

0.0 

• 

• 

• 

21 

• 

• 

319.8 

• 

• 

1670.4 

• 

• 

1315.0 

• 

• 

1670.4 

• 

• 

995.1 

• 

• 

0.0 

• 

• 

995.1 

• 

• 

5046.6 

• 

• 

1314.9 

22 

0.0 

125.2 

0.0 

801.4 

0.0 

0.0 

0.0 

125.2 

0.0 

23 

0.0 

0.0 

0.0 

0.0 

603.7 

0.0 

0.0 

0.0 

0.0 

24 

0.0 

13.2 

0.0 

39.7 

13.2 

0.0 

13.2 

0.0 

0.0 

25 

42.9 

0.0 

42.9 

0.0 

1115.1 

0.0 

0.0 

0.0 

0.0 

26 

0.0 

19.9 

0.0 

1054.1 

0.0 

0.0 

0.0 

0.0 

0.0 

27 

60.8 

243.2 

60.8 

1378.1 

729.6 

20.3 

121.6 

830.9 

830.9 

Note:  Jobs  equate  to  those  shown  in  Figure  4.  Representative  sites 
are  typical  organizational  units  with  a  characteristic  combination  of 
missions/  weapon  systems/  or  set  of  jobs,  used  as  a  basis  for  estimating 
average  training  capacities  and  costs  (see  Table  4  for  Site  1  training 
capacity  report). 
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would  then  generate  reports  for  comparison  with  the  data  from  a  baseline 
current  U  &  T  pattern  run;  differences  in  values  (annual  training  costs, 
total  AFS  population  in  future  years,  etc.)  would  reflect  the  relative 
inpact  of  the  change.  Another  type  of  change  would  be  to  change  the 
content  of  seme  course,  such  as  the  ABR,  and  then  run  the  system  to  assess 
the  changes  in  costs  and  total  COT  requirements  for  the  specialty. 

Any  major  proposed  change  in  a  specialty  should  be  dealt  with  as  a 
formal  alternative  U  &  T  pattern,  so  that  possible  consequences  can  be 
examined  in  some  detail  and  data  collected  which  describe  Air  Force 
managers'  preferences  among  a  set  of  alternatives.  Some  changes,  such  as 
a  merger  of  several  AFSs  at  the  technician  level  (as  preposed  in  RIVET 
WORKFORCE) ,  may  exceed  present  system  capacities  (unless  a  new  OSR  is 
accomplished  using  a  redeveloped  task  list  covering  all  specialties 
involved,  or  needed  data  are  estimated  in  some  other  way) . 

Most  of  the  possible  changes  which  might  be  considered  for  an  AFS  can 
be  modeled  by  changing  input  parameters,  course  content,  job  content,  or 
career  patterns  within  the  TDS.  Such  changes  are  processed  as  modeling 
runs  with  specified  values  of  selected  variables.  Analysis  then  focuses 
on  how  such  changes  impact  on  training  capacity  and  total  training  costs. 
The  training  capacity  reports  generated  in  this  process  will  also  high¬ 
light  any  constraints  on  training  capability  in  terms  of  training 
equipment,  instructor  availability,  or  other  significant  problems. 

An  example  may  serve  to  illustrate  how  potential  changes  can  be 
evaluated.  The  problem  might  be  a  resource  constraint  in  conducting  QJT 
in  some  units,  such  as  not  enough  hours  when  a  piece  of  test  equipment — a 
Weapons  Release  Control  System  (WRCS)  Analyzer  in  the  Radar  and  Inertial 
Navigation  Systems  Maintenance  specialty  (AFS  328X4) — is  available  for  use 
in  training.  One  approach  to  the  problem  would  be  to  move  training  of 
that  equipment  from  an  QJT  setting  to  a  formal  course  (an  FTD  or  the  basic 
resident  course  at  the  TIC) .  The  first  possibility  can  be  modeled  in  the 
TDS  by  adding  enough  hours  to  achieve  the  required  proficiency  (as 
indicated  by  the  Allocation  Curve  for  the  WRCS  TIM)  to  the  resident 
course.  A  second  model  would  be  to  add  hours  for  this  training  to  the  FTD 
course. 

The  results  of  these  analyses  are  shown  in  Table  6,  along  with  data 
from  the  current  U  &  T  pattern  as  a  baseline  for  comparison.  It  should  be 
noted  that  the  "Exceeded"  under  the  current  U  &  T  pattern  (first  column) 
indicates  that  some  resource  constraint  exists  (in  this  case,  an  equipment 
availability  constraint) .  Note  also  that  the  proposed  solution  of  adding 
the  required  WRCS  Analyzer  training  to  the  FTD  (third  column,  far  right) 
did  not  solve  the  problem;  CUT  capacity  is  still  exceeded.  Moving  the 
training  to  the  resident  (second  column,  middle)  course  does  appear  to 
solve  the  constraint  problem,  but  at  an  additional  ABR  cost  of  about 
$20,000.  There  is,  however,  some  offsetting  reduction  in  CUT  costs  since 
WRCS  Analyzer  training  was  removed.  In  this  particular  example,  the  costs 
calculate  out  to  be  about  the  same  but  providing  the  training  in  the  ABR 
does  not  exceed  OJT  capacity  of  units  in  the  field.  Obviously,  this  is  a 
feasible  and  practical  solution. 

The  point  here  is  that  once  a  problem  and  potential  solutions  have 
been  identified,  the  IOS  modeling  capability  can  be  used  to  translate  the 


Table  6.  Comparison  of  AFS  328X4  U  &  T  Patterns  Involving  Movement 
of  WCRS  Analyzer  Training  (AER  =  Airman  Basic  Resident; 

FTD  =  Field  Training  Detachment) 


CURRENT  U&T 
PATTERN 


ABR  COURSE  COSTS 
FTD  COURSE  COSTS 
TOTAL  COURSES 
COT  COSTS 
COT  CAPACITY 


$1,676,352 
$  45,647 
$2,724,296 
$5,096,500 
EXCEEDED 


AUGMENTED 
ABR  COURSE 

$1,696,406 

$  45,647 

$2,744,350 

$5,076,792 

NOT  EXCEEDED 


AUGMENTED 
FTD  COURSE 

$1,676,352 

$  53,495 

$2,732,145 

$5,095,981 

ITCCOTDKr) 


potential  solutions  into  modifications  of  the  specialty  data  base  (the 
current  U&T  files).  The  TDS  software  is  then  employed  to  generate 
products  for  each  potential  solution  and  results  can  be  compared  to 
baseline  data  (current  U&T  products)  by  Air  Force  decision  makers. 

2. 3. 4. 2  Optimization  Functions.  Given  the  almost  limitless  number  of 
possible  changes  Which  might  be  studied,  a  TDS  analyst  or  functional  user 
might  wish  to  take  another  approach  to  assessing  specialty  training 
changes.  This  approach  could  mate  use  of  the  IQS  optimization  software. 
The  analyst  or  user  can  specify  an  objective  function  or  goal  (such  as 
minimization  of  CUT  cost  or  total  training  costs,  or  maximization  of  the 
amount  of  available  equipment,  etc.),  run  the  optimization  program,  and 
examine  the  effects  on  the  specialty  if  the  objective  function  is 
maximized  or  minimized.  The  analyst  can  ask  "What  if"  questions;  for 
exanple: 


What  is  the  inpact  on  total  training  costs  if  we  minimize 
initial  resident  course  instruction? 

What  happens  to  specialty  jobs  (proficiency)  if  we  maximize 
FTD  training  and  minimize  QJT? 

What  is  the  inpact  on  proficiency  acquisition  and  training 
costs  _i_f  we  have  a  10%  cut  in  new  recruits  entering  training? 

Seme  potential  optimization  problems  may  become  visible  during 
modeling  runs  of  the  specialty  as  training  constraints  are  identified,  or 
possible  new  models  may  be  suggested  try  ini ticil  optimization  runs  (see 
Figure  12).  Other  possible  optimizations  will  be  suggested  by  general  Air 
Force  trends,  such  as  budget  cuts  or  changing  operational  priorities.  In 
some  cases,  these  could  be  complex  problems  with  severed  constraints  and 
multiple  values  to  be  optimized. 

The  approach  taken  in  employing  optimization  algorithms  to  solve 
maximization  or  minimization  problems  in  the  TDS  is  to  employ  modular  data 
bases  and  seek  solutions  at  the  lowest  possible  level.  This  isolates 
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>.  Relationship  of  Modeling  and  Optimization  Functions  of  the  IOS. 
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solutions  to  only  the  area  of  interest  and  has  considerable  efficiency  in 
terms  of  saving  computer  time.  Only  the  largest  optimization  problems, 
such  as  minimizing  total  specialty  training  costs,  would  require 
employment  of  the  entire  TDS  data  base.  More  limited  problems  can  thus  be 
dealt  with  by  limited  program  runs. 

The  computer  programs  used  to  provide  optimization  runs  were  designed 
so  that  they  can  be  interfaced  with  TDS  data  bases  in  the  IOS.  A  detailed 
review  of  such  software  is  beyond  the  scope  of  this  overview  report.  For 
details  of  constraints,  available  routines,  and  procedures  for  using  such 
software,  see  the  TDS  Procedural  Guide  (Vaughan,  Mitchell,  Marshall, 
Feldsott,  &  Rueter,  1988). 

2. 3.4. 3  User  Interface  Functions.  The  IOS  also  serves  as  an  interface 
with  TDS  users.  It  receives  all  requests,  calls  appropriate  data  from  the 
other  subsystems  or  TDS  files,  and  creates  products  to  meet  the  users' 
needs.  In  its  present  configuration,  the  interface  is  handled  through 
on-line  computer  requests  (control  cards,  specification  of  variables, 
creation  of  required  data  files)  and  the  resulting  products  (data  files 
and  reports) . 

At  present,  the  IOS  interface  is  structured  for  separate  modeling  and 
optimization  runs.  This  design  permits  selection  of  the  work  to  be  done 
without  requiring  that  both  processes  be.  engaged.  It  also  offers 
considerable  flexibility  in  that  the  user  can  select  the  desired  function 
and  complete  a  full  run  or  simply  engage  a  routine  for  additional  analysis 
as  needed.  This  modularization  is  a  necessary  efficiency  at  the  present 
stage  of  TDS  development.  Creation  of  fully  interactive  IOS  software  (to 
create  menus,  options  to  be  specified,  and  automated  report  formats)  at 
this  stage  would  be  counterproductive,  since  not  all  of  the  parameters  and 
desired  report  formats  are  known.  Rather,  this  aspect  of  IOS  user 
interface  software  must  be  deferred  until  after  completion  of  sensitivity 
analysis  and  the  full  Test  and  Evaluation  (T  &  E)  of  the  system. 


2.4  TDS  Software 

The  present  TDS  subsystems  software  packages  were  initially 
constructed  on  VAX  computers  at  McDonnell  Douglas  Astronautics  in  St. 
Louis  and  the  CONS  AD  Research  Corporation  in  Pittsburgh.  After  complete 
development  and  testing  on  these  systems,  TDS  software  was  transferred  to 
the  AFHRL  UNISYS  1100  system  for  testing,  debugging,  and  validation.  All 
TDS  software  was  written  in  Fortran  so  that  it  would  be  transportable 
between  computer  systems;  however,  some  reprogramming  proved  necessary  to 
accommodate  AFHRL-unique  FRISM  interface  programs  and  to  meet  AFHRL 
Information  Sciences  Division  standards. 

The  TDS  is  now  resident  on  the  AFHRL  UNISYS  computer  system  and  has 
undergone  initial  testing  and  validation.  Additional  testing  is  planned; 
the  present  contract  was  modified  to  include  a  sensitivity  analysis  to 
assess  the  impact  of  changes  in  various  types  of  data  on  system  output. 
In  addition,  the  contract  modification  required  planning  and  conducting  a 
full  test  and  evaluation  of  the  system.  These  activities  will  be 
summarized  in  a  separate  report. 
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3.  POTENTIAL  TDS  USES  AND  USERS 


The  TDS  is  designed  to  service  a  number  of  different  types  of  Air 
Force  decision  makers.  One  of  the  major  tasks  in  the  TDS  development 
effort  was  to  identify  potential  users  of  the  system  and  to  take  their 
interests,  requirements,  and  needs  into  account  in  planning  and  developing 
the  system  (see  Mitchell,  Sturdevant,  Vaughan,  and  Rueter,  1987).  This 
information  was  used  to  refine  the  TDS  preliminary  design  (Vaughan, 
Yadrick,  Perrin,  Cooley,  Dunteman,  Clark,  &  Rueter,  1984)  and  to  construct 
a  plan  for  transitioning  the  TDS  into  an  operational  Air  Force  program 
(Vaughan,  Yadrick,  Perrin,  Mitchell,  Sturdevant,  Rueter,  &  Ward,  1985). 
The  final  objective  of  these  efforts  was  to  create  a  draft  Air  Force 
Regulation  to  implement  the  system  (see  Appendix  A). 

This  chapter  briefly  reviews  the  potential  users  of  the  TDS  in  order 
to  establish  the  range  of  possibilities  for  employment  of  the  system.  (For 
a  discussion  of  the  visits,  conferences,  and  other  contacts  with  Air  Force 
personnel  and  organizations  on  which  this  information  is  based,  see 
Mitchell  et  al.,  1987.) 


3.1  TDS  Users 

A  number  of  potential  users  of  the  TDS  were  identified  and  their  needs 
considered  in  the  development  of  the  system. 

3.1.1  HO  US AF 

One  of  the  major  functions  of  the  TDS  is  to  provide  services  to  the 
Air  Staff  for  the  Planning,  Programming,  and  Budgeting  System  (PPBS). 
Currently,  such  planning  and  programming  focus  primarily  on  budgeting  for 
formal  technical  training  since  OJT  requirements  and  costs  are  not  visible 
in  the  system  (Rueter,  Bell,  &  Malloy,  1980).  Making  all  Air  Force 
specialty  training  requirements  visible  and  creating  realistic  estimates 
of  OJT  have  been  central  themes  in  the  development  of  the  TDS,  and  they 
have  been  fully  accommodated  in  the  design  of  the  FUS  and  RCS.  Such  data 
should  be  extremely  useful  for  HQ  USAF  functional  managers  and  training 
planning  staffs.  Alternative  U  &  T  patterns  allow  such  managers  to  consider 
major  planning  options.  The  ability  of  the  IOS  to  provide  data  estimates 
for  "what  if"  questions  should  be  invaluable  as  a  major  planning  tool. 

3.1.2  Training  Planning  Teams 

Another  anticipated  major  use  of  the  system  is  to  service  the  needs  of 
Training  Planning  Teams  (TPTs) ,  which  are  multi-command  teams  convened  by 
the  Air  Staff  (functional  manager  or  other  major  staff  element)  through 
issuance  of  appropriate  programming  documents  (see  AFR  50-8,  6  Aug  84, 
paragraph  3b).  The  TPT  process  is  a  recent  Air  Force  innovation  which  is 
still  being  evolved  (for  details  of  1984  -  1987  TPT  responsibilities  and 
participation,  see  Mitchell  et  al.,  1987,  pp.  32-45).  Currently,  a  draft 
revision  of  AFR  50-23  is  being  coordinated  within  Headquarters  USAF  and 
with  MAJCOMs  which  will  limit  TPTs  to  new  weapon  systems  acquisition 
projects  and  authorize  U  &  T  Workshops  as  the  primary  forum  for  specialty- 
specific  training  requirements  decisions. 
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To  effect  some  major  change  resulting  from  acquisition  of  a  new  weapon 
system,  TPTs  must  build  a  Training  Development  Plan  (TDP) .  The  initial 
step  in  building  such  a  plan  is  the  development  of  a  comprehensive 
description  of  present  U  &  T  patterns  for  the  Air  Force  specialties  involved 
(like  the  specialty  models  of  the  TDS).  TDS  cost  and  resource  estimation 
procedures  may  provide  a  way  to  realistically  evaluate  the  cost  implica¬ 
tions  of  a  preposed  change  to  individual  specialties.  In  addition,  the 
IOS  optimization  capabilities  could  provide  TPTs  with  the  ability  to 
visualize  and  evaluate  the  potential  consequences  of  their  decisions  on  a 
real-time  basis.  Thus,  the  TDS  could  help  a  TPT  make  more  realistic  and 
economically  viable  recommendations  for  major  changes  in  specialty 
training  and  utilization  programs. 

3.1.3  Air  Force  Specialty  Utilization  and  Training  Workshops 

U  &  TWs  are  routinely  convened  by  HQ  ATC  so  that  training  managers  can 
meet  with  user  command  representatives  to  review  present  utilization  and 
training  programs  (see  AFR  8-13).  Such  meetings  are  called  in  response  to 
user  command  requests  for  changes  in  training,  as  a  result  of  training 
evaluation  feedback,  or  upon  completion  of  a  new  occupational  survey. 
Typically  in  the  past  U  &  TW  meetings  have  focused  on  initial  skills 
training  programs  and  the  Specialty  Training  Standard  (STS)  for  a 
particular  Air  Force  specialty  or  set  of  related  specialties,  since 
detailed  information  on  other  training  programs  (FTDs,  MTTs,  and  OJT)  has 
not  been  readily  available  to  all  participants.  As  with  the  TPT  process, 
the  TDS  could  assist  U  &  TW  representatives  in  understanding  present 
training  and  assignment  patterns  and  in  properly  evaluating  various 
proposals  for  change.  [As  noted  earlier,  a  draft  AFR  50-23  will  upgrade 
U  &  TWs  as  the  primary  decision  making  process  for  individual  Air  Force 
specialties.  ] 

3.1.4  AFMPC  and  AFMEA 

In  addition  to  Air  Staff,  TPT,  and  U  &  TW  use,  the  TDS  is  also  intended 
for  use  by  Air  Force  Military  Personnel  Center  (AFMPC)  and  Air  Force 
Management  Engineering  Agency  (AFMEA)  planners  and  managers.  For  example, 
the  Classification  function  at  AFMPC  could  evaluate  recommended  AFR  39-1 
changes  in  terms  of  their  implications  for  AFS  jobs,  training  programs, 
and  related  costs  before  approving  or  rejecting  a  proposal.  AFMEA  might 
use  the  TDS  to  assess  the  effects  of  manpower  cuts  on  an  AFS,  or  to 
examine  job  reengineering  alternatives  for  reallocating  manpower 
authorizations  within  a  specialty.  These  potential  uses  for  the  TDS  are 
largely  conjectural;  additional  research  and  development  would  be  required 
to  explore  such  possible  classification  and  manpower  use  of  the  system. 

3.1.5  ATC  or  MAJCOM  Functional  and  Training  Managers 

Such  managers  could  use  the  system  to  evaluate  ideas  and  proposals,  to 
secure  needed  training  and  assignment  pattern  information,  or  to  obtain 
data  on  how  other  commands  and  agencies  conduct  training  in  a  specialty  or 
how  they  are  utilizing  AFS  personnel.  In  addition,  training  managers  and 
developers  at  ATC  Technical  Training  Centers,  MAJ COM-designated  training 
development  units,  or  FTD  remote  locations  could  address  the  system  for 
assistance  in  defining  training  requirements,  preparing  job  descriptions 
(defined  by  sets  of  TTMs),  and  determining  relative  training  costs. 
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The  USAFOMC  is  responsible  for  Weighted  Airman  Promotion  System  (WAPS) 
test  development,  the  Air  Force  occupational  analysis  (OA)  program. 
Military  Training  Standard  (MTS)  development  and  training  packages,  and 
providing  training  analysis  and  development  services  to  Technical  Training 
centers  and  other  users.  USAFOMC  provides  support  services  for  U  &  TWs  and 
TPTs,  and  has  been  responsible  for  developing  Training  Development  Plans 
(TDPs)  or  Training  Requirements  Analysis  (TRAs)  for  several  AFSs.  Some  of 
the  data  requirements  of  TDPs  and  TRAs  are  typical  TDS  products  (such  as 
the  current  U  &  T  pattern  description),  and  the  TDS  can  provide  considerable 
assistance  to  the  USAFOMC  in  servicing  this  type  of  function. 

Because  TDS  studies  begin  with  the  specialty  task  list  and  OS  data, 
the  TDS  could  provide  meaningful  feedback  to  the  OA  program,  and  perhaps 
suggest  further  potential  improvements  to  the  OA  process.  For  example, 
task  clustering  procedures  developed  for  the  TDS  have  already  been 
enhanced  (through  a  CODAP  improvement  project)  and  implemented  into  the  OA 
program  at  USAFOMC.  The  task  modules  defined  for  TDS  may  be  useful  in 
revising  the  organization  of  task  inventories.  Several  other  TDS 
innovations  (such  as  AFS  modeling)  have  considerable  potential  for  use  in 
the  operational  OA  program  (Mitchell,  Vaughan,  Yadrick,  &  Collins,  1987; 
Mitchell,  Vaughan,  Yadrick,  Collins,  &  Hernandez,  1988). 

3.1.7  The  Research  Community 

As  other  AFHRL  research  programs  come  to  fruition,  the  types  of  data 
provided  by  the  TDS  (task  clusters  or  TTMs,  U  St  T  patterns,  alternative 
training  possibilities,  training  cost  and  capacity  estimates,  etc.)  may  be 
applied  in  a  number  of  innovative  ways  to  help  improve  Air  Force  MPT 
programs.  The  TDS  can  provide  data  on  specialty  jobs  and  training 
requirements  (for  both  current  and  proposed  AFS  utilization  patterns)  and 
training  costs  which  may  be  needed  to  implement  the  Advanced  On-the-job 
Training  System  (AOTS);  in  turn,  AOTS  potentially  can  provide  new  types  of 
data  which  may  be  useful  in  future  TDS  development,  such  as  procedures  for 
determining  prerequisite  knowledge  requirements  or  new  task  proficiency 
measures.  In  addition,  the  TDS  itself  and  one  or  more  of  its  subsystems 
may  be  useful  research  tools  to  explore  the  effects  of  systematic  changes 
of  job -entry  aptitude  requirements,  student  flows,  or  other  parameters  on 
AFS  training  programs  and  job  content.  As  the  system  evolves,  additional 
possible  interactions  among  MPT  research  projects  may  emerge;  these 
efforts  could  be  made  synergistic  (Ruck  &  Mitchell,  1987). 


3.2  TDS  User  Interface 

As  developed  in  this  initial  R&D  effort,  the  TDS  is  designed  to  operate 
on  the  AFHRL  UNISYS  1100  computer,  and  thus  must  be  addressed  via  an  AFHRL 
task  scientist  or  system  manager.  The  system  is  presently  a  completed  R&D 
product  although,  as  noted  earlier,  it  will  undergo  extensive  test  and 
evaluation  as  well  as  sensitivity  analysis. 


The  system  may  eventually  be  addressable  to  some  authorized  users  via 
another  computer  system  which  could  interface  with  the  AFHRL  system.  In  the 
planned  follow-on  R&D,  an  operational  version  of  the  system  will  be  hosted 


on  another  system.  The  most  likely  candidate,  as  noted  in  the  TDS 
Transition  Plan  (Vaughan,  Yadrick,  Perrin,  Mitchell,  Sturdevant,  Rueter,  & 
Ward,  1985),  is  the  USAFOMC  IBM  computer  at  Randolph  AFB,  Texas.  Once  the 
system  is  installed  on  that  (or  an  equivalent)  system,  major  users  could 
be  granted  access  via  a  dial-up  capability  using  a  modem  and  remote 
computer  terminal.  Such  service  would,  however,  be  contingent  on  the 
development  of  additional  software  to  support  remote  terminal  operations. 


4.  DISCUSSION  OF  ISSUES 


The  research  and  development  resulting  in  a  proof-of-concept  Training 
Decisions  System  (TDS)  has  been  completed.  Information  was  gathered  from 
a  variety  of  potential  TDS  users,  and  their  concerns  and  interests  have 
been  incorporated  into  the  system  design  where  possible.  The  system  used 
occupational  survey  (OS)  data  as  a  starting  point  to  cluster  tasks  into 
Task  Training  Modules  (TT'Ms)  which  were  then  used  to  describe  jobs  and 
training  programs  of  an  Air  Force  specialty  and  serve  as  the  basis  for 
collecting  additional  data.  New  data  collection  instruments  were 
developed  to  gather  information  about  where  TTMs  are  trained,  where  they 
could  be  trained,  training  time,  and  other  variables.  Specialties  were 
modeled,  alternatives  were  defined,  and  managers'  preferences  among 
various  utilization  and  training  (U  &  T)  patterns  were  collected  for 
analysis.  Training  costs  and  resource  requirements  information  were 
generated,  including  very  comprehensive  estimates  of  on-the-job  training 
costs  and  capacities  for  representive  field  units.  Comparisons  were  made 
among  alternative  U  &  T  patterns  in  terms  of  total  training  costs  and 
capacities  which  very  clearly  highlight  the  possible  consequences  of 
decisions  by  specialty  managers  and  training  decision  makers.  Overall, 
the  proof-of-concept  TDS  gives  such  managers  and  decision  makers  a 
significantly  enhanced  capability  to  visualize  and  evaluate  the  results  of 
their  own  decisions. 

As  in  any  significant  R&D  effort,  some  issues  and  problems  were 
encountered  during  the  development  of  the  TDS.  This  chapter  notes  the 
issues,  documents  some  of  the  alternative  methodologies  tested  to  overcome 
the  problems  encountered,  and  outlines  potential  future  lines  of  research 
and  development  of  the  system. 


4.1  General  Issues 

There  are  severed,  genereil  topics  (involving  more  than  one  TDS 
subsystem)  which  warrant  some  discussion.  Subsystem  issues  will  be 
discussed  in  the  following  section 

4.1.1  Subject-Matter  Expert  (SME)  Involvement  and  Availability 

The  TDS  project  was  designed  to  make  extensive  use  of  SMEs  as  primary 
sources  of  information  and  estimation  of  specialty  data.  This  approach 
has  been  used  very  successfully  in  prior  AFHRL  research  and  development 
efforts  such  as  the  validation  of  the  Weighted  Airman  Promotion  System 
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(WAPS),  occupational  analysis  research,  task  and  occupational  difficulty 
benchmarking  (to  set  aptitude  requirements),  training  emphasis  research, 
strength  and  stamina  requirements  analysis,  safety  priorities 
determinations,  and  a  number  of  other  MPT  projects. 

Some  of  the  most  successful  development  activities  in  the  TDS  project 
were;  accomplished  using  this  approach.  For  example,  the  initial  task 
clustering  effort  was  undertaken  with  AFS  811XX  using  a  large  panel  (20+) 
of  SMEs  at  a  meeting  at  the  Security  Police  Academy,  Lackland  AFB,  Texas, 
in  May  1985.  This  effort  was  made  possible  through  the  strong  support  of 
the  functional  manager  (Air  Force  Office  of  Security  Police,  Kirtland  AFB, 
New  Mexico)  who  requested  subordinate  units,  including  the  Security  Police 
Academy,  to  provide  SMEs  for  a  4-day  session.  Other  data  collecting 
efforts  were  integrated  whenever  possible  with  ongoing  TPT  and  U  &  TW 
activities  scheduled  for  other  purposes;  this  greatly  facilitated  the 
development  of  needed  TDS  data  bases. 

TDY  funds  for  SME  participation  in  research  projects  are  very  limited 
both  by  budgetary  constraints  and  by  the  difficulty  in  forecasting  just 
when  in  an  R&D  project  such  participation  will  be  necessary.  In  some 
cases  involving  very  specialized  or  equipment-specific  tasks,  only  a  very 
small  number  of  SMEs  have  the  knowledge  and  experience  required  to  make 
professional  judgments  or  estimates;  often  such  specialized  individuals 
are  fully  involved  in  operational  requirements  and  cannot  easily  be  made 
available  for  R&D  participation. 

Consequently,  there  were  instances  where  alternative  approaches  had  to 
be  used  in  the  IBS  development  effort.  TDS  researchers  also  visited 
successive  small  groups  of  SMEs  at  their  own  bases  or  other  research  sites 
(AFHRL  Detachment,  Bergstrom  AFB,  TX).  Extensive  use  was  made  of 
telephone  contacts  with  training  SMEs,  particularly  those  in  FTD  units  at 
widely  dispersed  operational  bases.  Hie  AFHRL  staff  also  assisted  by 
performing  some  data  collection  trips  to  additional  bases  to  secure  needed 
RCS  data.  The  research  team  then  had  to  integrate  the  results  of  the 
various  data  gathering  trips  or  contacts  to  approximate  consensus  results. 

The  occasional  limitation  of  face-to-face  SME  interaction  precluded 
use  of  desired  techniques  (such  as  full  consensus  ratings,  SME  validation 
of  survey  data  summaries,  and  SME  group  estimation  of  the  impact  of 
changes  on  specialty  jobs,  training  courses,  and  other  programs).  This 
kind  of  limitation  can  perhaps  be  solved  by  combining  TCS,  FUS,  and  RCS 
data  collection  efforts  into  one  or  two  major  meetings  and  securing  full 
functional  manager  support  (as  was  done  with  Security  Police). 

In  an  operational  setting,  SME  availability  might  be  assured  through 
extensive  MAJCOM  and  functional  manager  coordination  and  periodic  meetings 
of  U  &  TWs.  In  future  R&D  programs,  however,  sufficient  TDY  funds  should  be 
programmed  to  ensure  that  adequate  numbers  of  representative  SMEs  are 
available,  and  full  support  obtained  from  HQ  USAF  and  MAJCOM  functional 
managers  for  the  specialty  or  specialties  involved. 

4.1.2  Assessing  System  Products — The  Criterion  Issue 

A  major  innovation  in  the  TDS  was  the  development  of  data  collection 
methods,  procedures,  and  software  to  generate  estimates  of  specialty 
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training  costs  and  capacities.  Econometric  modeling  was  blended  with 
occupational  analysis  outcomes  and  new  data  collection  or  estimation 
techniques  to  examine  training  costs,  including  costs  of  OJT,  at  a  greater 
level  of  detail  than  has  ever  been  possible  previously. 

The  various  pieces  of  TDS  serve  their  individual  functions  well  and 
have  been  tested  or  validated  against  known  data  sources  where  possible. 
However,  when  these  pieces  are  combined  in  the  TDS  to  generate  total 
training  cost  and  capacity  estimates,  there  are  no  hard  criteria  against 
which  the  validity  of  the  totals  can  be  assessed  for  most  training  states 
(and  particularly  OJT  costs  for  particular  jobs  or  mission  organizations). 

One  approach  to  testing  overall  system  operation  is  to  check  the 
validity  of  data  against  known  sources,  for  at  least  some  subset  of 
information.  Work  is  underway  to  check  some  job  assignment  flow  and 
specialty  attrition  estimates  against  rates  calculated  from  Uniform  Airman 
Records  (UAR)  data  at  three  points  in  time  (at  24-month  intervals) .  This 
approach  is  expensive  in  terms  of  custom  programming  and  computer  time  but 
provides  some  validation  of  at  least  some  segments  of  the  TDS  data  base. 

The  limitations  created  by  the  lack  of  an  overall  criterion  can  be 
mitigated  to  some  degree  by  extensive  testing  of  the  system  using  varying 
data  estimates  or  run  parameters.  This  work  in  being  undertaken  through 
the  sensitivity  analyses  noted  earlier.  Such  sensitivity  analyses  are 
aimed  at  determining  the  relative  impact  of  various  data  sources.  If  the 
system  is  run  with  a  quick  and  inexpensive  estimate  and  results  compared 
to  those  from  a  time-consuming,  expensive  estimate  and  there  is  no 
practically  significant  impact  on  the  final  output,  then  the  quicker  and 
less  expensive  estimate  is  to  be  preferred.  Where  the  use  of  a  more 
costly  estimation  procedure  does  make  a  difference,  then  it  must  be  used 
for  the  system  to  operate  properly,  or  further  R&D  must  identify  or 
develop  a  better  (less  expensive)  way  to  collect  the  information. 

4.1.3  Complexity  of  Specialty  Models  and  the  TDS 

The  proof-of-concept  TDS  is  an  extremely  complex  system,  requiring  a 
large  variety  of  types  of  input  data,  including  parameter  specifications, 
existing  data  from  established  data  sources,  and  new  data  collected  from 
specialty  SMEs  in  the  field.  The  various  subsystems  are  all  unique, 
innovative  sets  of  both  procedures  and  software,  designed  specifically  to 
systematically  build  a  body  of  information  and  data  files  concerning  a 
specialty  to  a  much  more  complex  level  than  has  previously  been 
accomplished.  The  outputs  of  these  subsystems  are  then  combined  to 
serve  the  needs  of  Air  Force  decision  makers  who  need  concise,  specific 
data  summaries  and  comparisons. 

As  presently  configured,  the  TDS  is  a  complex  system  which  must  be 
operated  by  an  individual  or  team  with  extensive  background  in  both 
statistical  modeling  and  Air  Force  operational  information.  This  is  to 
be  expected  to  some  degree  with  any  experimental  system  designed  to 
function  in  an  RSD  environment.  As  Air  Force  personnel  become  familiar 
with  the  TDS,  operational  simplifications  can  be  found  and  implemented. 

In  another  sense,  the  TDS  is  not  complex  enough.  There  are  a  number 
of  aceaa  where  compromises  had  to  be  made.  For  example,  U  &  T  patterns  had 
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to  be  simplified  in  order  to  communicate  them  easily,  and  not  all  training 
programs  were  included  (if  the  number  of  participants  was  very  small  or 
the  programs  were  used  only  irregularly).  Some  assumptions  had  to  be  made 
in  order  to  formulate  information  in  ways  that  facilitated  its  use  in  the 
mathematical  simulation  of  incumbent  job  and  training  flows.  Some 
arbitrary  decisions  were  made  in  calculating  student  travel  costs  and 
other  variables,  since  only  representative  bases  are  included  in  the  model 
(as  opposed  to  a  census  of  every  base  utilizing  the  specialty). 

For  at  least  the  first  few  years  of  TDS  use,  its  operation  needs  to 
remain  in  the  hands  of  trained  system  specialists.  These  should  be 
individuals  with  a  good  background  in  ASCII  CO  DAP  operations,  job  typing 
and  analysis,  statistical  modeling,  econometrics,  and  decision  theory,  as 
well  as  data  collection,  data  analysis,  and  report  writing.  It  may  be 
that  a  team  of  individuals  extensively  trained  on  the  TDS  will  be  needed, 
to  ensure  that  all  the  requisite  skills  are  available.  At  this 
proof-of-concept  stage,  attempts  to  operate  the  TDS  using  only  the 
published  reports  and  software  documentation  would  be  counterproductive. 

Secondly,  whenever  the  system  is  used,  data  outputs  need  to  be  checked 
carefully  by  analysts  and  by  SMEs.  Specialty  SMEs  need  to  be  periodically 
asked  to  review  models  developed  for  their  career  field,  as  well  as  to 
review  data  outputs  for  realism  and  practicality.  Such  SMEs  would  need  to 
be  thoroughly  briefed  on  the  objectives  and  methodologies  used  in  the  TDS, 
and  must  be  reassured  that  in  the  TDS,  system  outputs  are  still  "research" 
products. 

Thirdly,  the  ideal  use  and  testing  of  the  TDS  would  be  to  extend  the 
use  of  the  system  to  some  additional  deliberately  chosen  specialties  where 
sequential  U  &  TWs  can  be  undertaken  in  parallel  with  the  TDS  study.  This 
would  provide  for  "real-world"  testing  of  TDS  data,  products,  and  possible 
decision  recommendations  until  the  system  is  fully  operational  and 
implemented.  Such  parallel  developments  would  also  result  in  suggestions 
for  improvements  and  further  innovations  which  could  make  TDS  more 
valuable  to  potential  users. 

4.1.4  Currency  of  Specialty  Data  in  TDS  Files 

One  issue  raised  by  some  HQ  ATC  Training  Staff  Officers  (TSOs) 
participating  in  TDS  progress  reviews  or  in  a  special  TSO  briefing  on  the 
TDS  Simulator  in  early  1987  was  the  question  of  who  will  maintain  and 
update  data  files  used  in  the  TDS.  There  are  changes  occurring  in  most 
specialties  constantly,  some  minor  and  some  with  significant  implications 
for  training.  The  TSOs  were  concerned  that  unless  ongoing  changes  were 
monitored  and  their  potential  impact  assessed,  the  TDS  data  base  could 
quickly  become  obsolete,  and  therefore  be  of  limited  utility  in  training 
decision  making.  The  implication  of  the  TSO  concern  is  that  if 
specialty-specific  data  bases  are  developed,  they  must  also  be  updated 
and,  where  necessary,  new  TDS  data  must  be  collected.  This  is  necessary 
if  TDS  is  to  be  useful  to  U  &  TWs,  TPTs,  and  training  staffing  agents. 
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4.2.1.  TCS  Issues 


In  the  development  and  validation  of  the  Task  Clustering  and  TTM 
Allocation  components  of  the  TCS,  severed  issues  were  encountered  which 
require  some  discussion.  Some  of  these  involve  the  nature  of  tasks  and 
TTMs  and  their  relative  merit  as  training  decision  aids. 

4. 2. 1.1  Task  Modules  as  Training  Modules.  Development  of  TTMs  through 
the  TDS  task  clustering  procedure  (co-performance  clustering  with  naming 
and  refinement  by  SMEs)  worked  well  for  the  four  specialties  studied  in 
this  R&D ;  however,  based  on  their  analysis  of  a  few  Security  and  Law 
Enforcement  (AFS  811XX)  TTMs,  some  USAFOMC  personnel  indicated  they  would 
prefer  that  TTMs  be  retitled  as  simply  "task  modules. "  This  would 
eliminate  the  inadvertent  connotation  that  all  tasks  within  a  task  cluster 
must  always  be  trained  together  (in  the  same  course  block).  Rather,  in 
TDS  most  TTMs  are  expected  to  be  trained  to  some  degree  of  proficiency  in 
several  different  training  settings. 

This  is  an  important  issue  for  TDS  in  terms  of  user  understanding  and 
use  of  the  system.  TTMs  must  have  a  general  acceptance  by  the  using 
community  (trainers,  managers,  and  field  personnel)  in  order  to 
effectively  meet  their  purpose  and  properly  function  as  a  decision  aid. 

The  suggestion  to  use  the  term  task  module  (TM)  rather  than  TTM  is  a 
very  constructive  idea  which  should  be  adopted.  In  the  TDS,  TTMs  are  used 
to  describe  both  jobs  and  training  programs;  TM  is  more  descriptive  of 
such  multiple  uses. 

4.2.1.2  PME  Versus  Technical  Tasks.  In  all  four  TDS  specialities,  some 
SMEs  involved  in  task  clustering  made  a  marked  distinction  between 
supervisory  and  management  task  groupings  (which  they  considered  the 
province  of  PME)  and  technical  task  clusters  (which  involve  technical 
training  or  OJT).  This  clearcut  dichotomy  is  consistent  with  the  current 
Air  Force  philosophy  regarding  PME.  Such  a  philosophy  constrains  AFS 
decision  makers  to  accept  this  dichotomy,  regardless  of  whether  the  needs 
of  each  AFS  are  properly  serviced  by  standard  Air  Force  PME  programs. 
More  than  one  participant  at  TDS  progress  reviews  suggested  that  standard 
PME  programs  do  not  fully  meet  the  needs  of  Communications/Electronics 
NCOs.  The  time-phasing  and  opportunity  to  attend  standard  PME  courses 
do  appear  to  differ  among  specialties.  For  the  TDS,  a  standard  set  of 
PME  TTMs  may  be  required  to  service  all  specialties,  or  a  better  way  needs 
to  be  developed  to  relate  PME  knowledges  and  skills  to  the  tasks  (TMs)  of 
each  specialty  so  as  to  identify  AFS -unique  requirements. 

4.2. 1.3  Principles  or  Prerequisite  Knowledges.  Current  technical 
training  programs  often  include  instruction  on  basic  principles  which  can 
be  linked  only  indirectly  to  specialty  tasks.  Examples  include 
electronics  principles,  introductory  medical  knowledge,  basic  mathematics 
skills,  etc.  It  is  difficult  to  account  for  such  prerequisite  knowledge 
training  or  to  estimate  the  degree  of  required  TTM  proficiency 
attributable  to  such  basic  knowledges  in  the  TDS  model.  This  problem  was 
addressed  in  the  present  TDS  effort  by  identifying  all  TTMs  which  may 
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require,  for  example,  electronics  knowledge,  and  then  attributing  equal 
portions  of  the  total  course  hours  (from  AFR  50-5)  to  those  TTMs.  Further 
R&D  effort  is  needed,  however,  to  develop  better  ways  of  accounting  for 
knowledge  fundamentals  in  the  system.  Separate  knowledge  TTMs  may  need  to 
be  constructed,  or  perhaps  the  AFS  task  list  itself  needs  to  be  modified 
to  include  knowledge-specific  basic  tasks. 

4. 2. 1.4  Weapon  System  Specific  Tasks.  In  some  specialties,  such  as 
Environmental  Systems  Maintenance  (AFS  423X1),  it  was  difficult  to  distin¬ 
guish  work  on  various  aircraft  systems,  due  to  the  generic  task  statements 
in  the  USAF  Job  Inventory.  Such  generic  tasks  may  disguise  system 
differences  in  task  performance  and  thus,  in  OJT  requirements.  If  this  is 
so,  then  TDS  would  significantly  underestimate  the  total  training  require¬ 
ments  and  costs  for  the  specialty.  Further  R&D  is  needed  to  assess  when 
tasks  should  be  more  weapon-system-specific.  Such  an  effort  has 
significant  implications  for  other  AFHRL  programs,  including  aptitude 
requirements  (task  difficulty),  person-job  match,  and  basic  skills 
research  and  development. 

4. 2. 1.5  Concept  of  Proficiency.  Early  in  the  TDS  project,  the 
question  arose  of  how  to  address  the  level  of  proficiency  required  for 
each  task  or  set  of  tasks  (TTM).  Specialty  Training  Standard  (STS) 
proficiency  codes  appeared  ill-suited  for  collecting  quantitative 
estimates  of  required  proficiency;  some  type  of  scaled  values  were 
needed.  Secondly,  STS  codes  have  been  a  continuing  source  of  controversy 
between  the  resident  training  community  and  OJT  supervisors  (who  operate 
on  a  "go/no  go”  concept  of  task  proficiency). 

For  the  TDS,  the  OJT  go/no  go  concept  provided  a  good  starting  point 
for  an  operational  definition  of  required  proficiency.  The  "go”  point 
represents  the  target  level  of  performance  for  all  training;  if  an 
individual  reaches  that  point,  the  training  requirement  is  satisfied.  For 
TDS  purposes,  common  scale  values  can  be  assigned  by  dealing  in  terms  of 
the  preportion  of  the  required  proficiency  met  in  any  training  setting; 
that  is,  as  a  percentage.  Thus,  based  on  the  go/no  go  OJT  concept,  TDS 
assesses  the  required  proficiency  on  a  TTM  which  is  to  be  achieved  through 
training.  This  conceptualization  worked  extremely  well  in  the  TDS  project 
and  SMEs  quickly  understood  and  used  this  operational  definition  in  making 
training  requirement  estimates  and  allocations. 

At  some  point  in  the  future,  however,  this  construct  needs  to  be 
related  to  STS  proficiency  codes,  and  an  equating  or  translation  scheme 
devised.  This  is  needed  to  properly  utilize  TDS  data  in  U  &  TWs  for  STS 
reviews.  In  addition,  further  R&D  is  desirable  to  investigate  advanced 
proficiency  levels  which  go  beyond  basic  proficiency  requirements  (i.e., 
advanced  troubleshooting,  highly  skilled  task  performance,  etc.),  except 
where  such  advanced  activities  are  represented  in  a  separate  task.  This 
is  an  issue  particularly  relevant  for  higher  skill  and  grade  levels. 

4.2. 1.6  TTM  Allocation  Data.  In  the  present  TDS  R&D,  there  was 
occasionally  some  inconsistency  between  information  gathered  in  the  TTM 
Allocation  Survey  and  the  data  derived  for  the  RCS.  In  some  cases, 
training  resource  requirements  or  availability  information  was  not 
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provided  by  the  SMEs  for  some  TTMs  which  other  SMEs,  responding  to  the 
Allocation  Survey,  indicated  were  trained.  It  is  not  clear  if  this  was  a 
data  collection  problem,  if  it  was  a  sampling  issue,  or  if  one  or  the 
other  set  of  SMEs  simply  misunderstood  directions.  In  any  case,  data 
inconsistency  is  a  problem  which  needs  to  be  resolved.  One  solution  would 
be  to  integrate  the  data  collection  for  the  TIM  Allocation  and  the  RCS 
surveys,  using  exactly  the  same  sample  of  SMEs.  Another  possible 
alternative  would  be  to  have  such  discrepancies  resolved  to  consensus  in  an 
SME  panel  session  such  as  a  TPT  or  U  &  1VJ. 

4.2.2  FUS  Issues 

Several  issues  surfaced  during  the  development  of  current  and 
alternative  AFS  models  for  the  TDS. 

4.2.2. 1  Missing  Data  on  Training  Courses.  As  noted  earlier,  some 
training  courses  were  excluded  from  the  TDS  models  of  several  AFSs.  The 
types  of  courses  omitted  were  those  for  which  no  data  were  available, 
where  the  course  had  no  regular  input  (courses  on  standby),  or  courses 
where  the  number  of  students  involved  was  extremely  small.  In  the  proof- 
of- concept  TDS,  such  training  requirements  are  automatically  transformed 
into  an  COT  requirement  and  thus  are  accounted  for  in  the  system,  because 
of  the  way  OJT  needs  are  computed.  However,  the  allocation  data  for 
several  specialties  suggest  that  some  relevant  TTMs  cannot  be  trained  to 
complete  proficiency  through  OJT  alone.  Thus,  the  simplified  or 
abstracted  model  developed  for  such  a  specialty  somewhat  overstates  the 
need  for  OJT  and  underestimates  the  requirement  for  formal  classroom 
training.  Although  the  degree  of  distortion  is  relatively  small,  this  is 
a  problem  which  needs  to  be  examined  through  further  R&D.  If  it  is  found 
to  be  a  significant  problem,  then  more  complex  AFS  models  may  be 
necessary. 

4. 2. 2. 2  Additional  Training  Settings.  In  attempting  to  build  a 
general  methodology  for  modeling  specialty  jobs  and  training,  some 
simplification  of  the  possible  training  settings  was  necessary. 
Generally,  the  TDS  deals  with  classroom,  hands-on  (field  training), 
self-study  (CDC,  etc.),  and  OJT.  Almost  all  Air  Force  training  can  be 
forced  into  these  generic  categories.  However,  such  a  simplified  schema 
does  not  provide  separate  data  for  a  number  of  other  possible  ways  in 
which  training  can  be  delivered.  Under  the  present  schema,  a  computer- 
delivered  training  system  (CDTS)  or  interactive  video  disk  (IVD)  in  a  unit 
learning  center  could  be  classified  as  either  classroom  training  or  as 
self-study.  Specialty-unique  training  systems,  such  as  some  of  those 
developed  by  Air  Force  Communications  Command  (AFCC)  or  the  Security 
Police  Educational  Subject  Block  Indices  (ESBIs),  are  well-developed 
systems  and  may  need  to  be  accounted  for  separately  in  the  modeling  of 
relevant  specialties.  For  each  additional  training  setting  added  to  the 
model,  additional  ratings  (allocations,  resource  requirements,  resource 
availability,  etc.)  would  have  to  be  generated,  further  complicating  an 
already  complex  system.  It  is  not  yet  dear,  based  on  work  with  just  four 
specialties,  whether  this  is  really  a  problem  or  if  the  simplified 
training  setting  schema  provides  sufficient  data  for  AFS  decision  makers. 
As  the  TDS  is  used  with  other  specialties  or  when  a  specialty  is  examined 
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a  second  time,  it  may  be  necessary  to  expand  the  training  settings 
considered  in  order  to  account  for  some  additional  training  delivery 
options,  even  at  the  risk  of  making  the  AFS  models  more  complex. 

4. 2. 2. 3  Steady-State  Versus  Variable  Input  Models.  One  of  the 
simplifying  assumptions  made  during  development  of  software  for  the  FUS 
was  that  of  a  constant  input  level.  In  the  present  FUS  simulation 
program,  the  rate  of  input  to  a  specialty  is  a  specified  value  and  the 
program  builds  up  a  specialty  population  over  an  extended  time  period 
until  the  value  reaches  an  equilibrium  which  approximates  the  present 
population  level.  Only  the  data  relating  to  this  equilibrium  period  are 
relevant  for  analysis  of  specialty  training  and  utilization  programs. 
This  kind  of  simplifying  assumption  was  necessary  in  the  development  of 
the  TDS  to  make  it  possible  to  produce  the  simulation  model.  However, 
examination  of  historical  data  from  ATC  production  records  and  from  the 
Occupational  Research  Data  Bank  (ORDB)  at  AFHRL  strongly  suggests  that 
most  specialties  have  varied  considerably  in  input  over  the  last  6  to  8 
years.  Indeed,  there  appear  to  be  some  general  trends  across  time  which 
should  be  reflected  in  any  specialty  model.  For  example,  the  input  for 
AFS  423X1,  Environmental  Systems  Maintenance,  has  decreased  through  the 
years  as  some  bases  convert  to  contract  maintenance  for  such  equipment 
(mostly  ATC  bases  or  for  C-9  aircraft).  In  Security  Police  (AFS  811XX), 
there  has  been  a  gradual  buildup  in  recent  years  (the  TPR  has  increased 
from  5,000  to  7,000+)  as  ground  launched  cruise  missiles  (GLCMs)  were 
deployed  in  Europe  and  the  Space  Command  complex  was  activated;  this  trend 
was  reversed  when  the  Intermediate  Range  Missile  treaty  was  signed  at  the 
1988  Moscow  summit.  Such  trends  in  two  of  the  four  specialties  included 
in  the  present  R&D  are  sufficient  to  indicate  that  variable  input  modeling 
is  probably  required  so  that  the  TDS  can  better  approximate  general 
trends.  This  will  require  some  reprogramming  to  allow  more  complex 
simulations;  this  possibility  should  be  considered  for  any  follow-on  TDS 
project. 

4. 2. 2. 4  Training  Decay  Functions.  With  good  training,  a  person  can 
achieve  full  proficiency  in  performing  a  group  of  tasks  (TTM) ;  however, 
unless  the  individual  regularly  performs  the  tasks,  he  or  she  will  tend  to 
lose  that  proficiency.  The  loss  rate  can  be  tested  and  displayed  as  a 
mathematical  function — the  training  decay  function  (TDF).  Such  a  function 
has  obvious  utility  for  TDS,  in  that  it  directly  affects  the  amount  of 
training  required  when  an  individual  moves  to  a  new  job  or  is  assigned  new 
responsibilities.  If  a  person  no  longer  has  full  proficiency  for  the  new 
tasks  due  to  the  passage  of  time  since  receiving  training,  or  has  not 
performed  the  tasks  before,  then  additional  training  is  necessary.  In  the 
TDS  development  effort,  this  problem  was  handled  by  assuming  that  if  a  TTM 
was  performed  in  the  previous  job  it  did  not  need  to  be  trained,  but  that 
if  it  was  not  performed  in  the  previous  job,  it  did  need  to  be  trained. 

There  is  a  good  body  of  scientific  literature  on  TDFs,  but  very  little 
information  is  available  on  such  functions  as  they  relate  to  specific 
tasks  or  groups  of  related  tasks  (TTMs) .  There  are  a  number  of  studies  in 
progress  on  the  value  of  prior  experience  and  how  knowledge  of  some 
subjects  decays  over  time.  As  such  information  becomes  available,  the 
TDFs  for  relevant  subject  areas  should  be  included  in  TDS  modeling  (if  the 
subject  areas  can  be  directly  related  to  TTMs).  Specific  R&D  to  identify 
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such  functions  for  Air  Force  training  areas  and  AFS -specific  TTMs  would  be 
worth  while. 

4.2.3  RCS  Issues 

Several  areas  of  the  RCS  included  problems  or  issues  which  merit 
discussion. 

4. 2. 3.1  Resource  Requirements  and  Representative  Sites.  The 
identification  of  the  amounts  of  resources  required  for  training  each  TTM 
by  each  training  setting  became  extremely  complex  and  difficult. 
Generalized  resource  requirements  surveys  were  used  which  listed  all  TTMs 
and  the  general  types  of  resources  expected  to  be  involved;  and  SMEs  were 
asked  to  estimate  quantities,  hours  of  student  use,  and  additional 
resources.  Trainers  of  specific  TTC  or  FTD  courses  were  also  to  indicate 
which  resources  were  shared  with  other  courses,  with  other  specialties,  or 
with  operational  users.  The  data  collected  indicated  that  some  SMEs  had 
difficulty  dealing  with  such  multiple  taskings.  In  addition,  data 
collected  in  this  survey  proved  inconsistent  with  information  developed  in 
later  Resource  Availability  surveys. 

As  noted  earlier,  some  improvement  in  the  quality  of  data  is  expected 
through  improved  instructions  and  changes  in  data  collection  procedures. 
However,  it  may  be  that  a  tailored  approach  to  this  kind  of  data 
collection  is  required.  If  representative  sites  can  be  identified  early 
(perhaps  in  the  initial  task  list  development  process),  then  the  TTMs  and 
resources  at  each  base  could  be  compiled  into  a  separate  booklet  so  that 
survey  respondents  at  that  base  need  deal  only  with  relevant  issues.  This 
would  necessitate  a  tailored  booklet  for  each  representative  site  but 
would  greatly  ease  the  time  and  effort  required  of  SMEs. 

4. 2. 3. 2  Resource  Availability.  There  was  some  disparity  between 
information  collected  in  the  TCS  Allocation  Survey  and  that  compiled  later 
from  the  RCS  Resource  Availability  questionnaire.  Part  of  this  problem 
may  be  due  to  confusion  concerning  the  concept  of  "training  time"  as  it  is 
used  in  the  TCS  (hours  spent  training  the  tasks  of  a  TTM)  versus  in  the 
RCS  (hours  during  which  specified  resources  might  be  used  for  training). 

A  further  anomaly  involved  disparities  between  the  resource  require¬ 
ments  and  resource  availability  data.  In  some  cases,  equipment  or 
supplies  were  indicated  as  required,  yet  none  was  shown  as  available  (even 
though  the  training  was  being  provided  successfully).  This  kind  of 
discrepancy  limited  the  extent  to  which  training  capacity  could  be 
calculated  for  the  test  AFSs  in  this  present  project.  Many  non-labor 
resources  had  to  be  excluded  from  the  calculations  at  this  time,  although 
the  capability  to  produce  such  estimates  does  exist  in  the  system. 
However,  a  better  way  to  collect  the  information  must  be  implemented,  or 
at  least  some  methodology  must  be  developed  which  permits  resolving 
inconsistencies  between  the  various  data  sets. 

As  an  interim  solution,  the  collection  of  training  allocation  data, 
resource  requirements  information,  and  resource  availability  data  should 
be  linked;  the  three  data  sets  should  be  gathered  at  the  same  time  with 
the  same  set  of  SMEs.  This  will  at  least  control  for  differences  in 
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survey  administration  and  ensure  that  a  common  set  of  definitions  is 
used.  One  possible  alternative  is  to  have  the  data  sets  that  are 
collected  by  survey  reviewed  by  a  panel  of  representative  SMEs  and  any 
disparity  resolved  through  face-to-face  negotiation.  This  is  how  such 
problems  would  probably  be  handled  in  an  operational  system;  i.e.,  through 
a  training  conference  or  U  &  TW. 

4. 2. 3. 3  RCS  Parameters.  In  some  cases,  decisions  had  to  be  made  as  to 
how  to  calculate  variable  costs  for  items  such  as  average  TDY  travel 
costs;  salary  costs  of  students  and  instructors  (estimated  from  average 
grade  levels);  student-to-instructor  ratios  in  classroom,  FTD,  and  OJT 
programs;  etc.  Such  decisions  were  embodied  in  separate  data  files  or 
parameter  specifications.  This  modular  approach  provides  considerable 
flexibity  for  the  TDS  in  that  as  such  rates  change  or  as  better 
estimation  procedures  are  developed,  new  values  can  be  inserted  into  RCS 
data  files  without  reengineering  the  system  (for  example,  as  TDY  travel 
and  per  diem  allowances  change,  one  could  revise  that  file  and  then 
calculate  the  impact  of  that  change  by  running  the  data  for  a  sample 
AFS).  All  of  the  parameters  and  estimation  methods  used  in  the  present 
TDS  are  candidates  for  improvement.  The  programmed  sensitivity  analysis 
and  T  &  E  of  the  system  will  identify  those  parameters  or  estimation  methods 
which  should  have  priority  for  further  development  work. 


4.2.4  IQS  Issues 

Some  areas  of  needed  improvement  in  IOS  modeling  activities  were 
addressed  earlier  (under  the  FUS;  see  section  4.2.2).  Other  IOS  issues 
involve  the  interface  with  users.  At  present,  the  IOS  has  separate 
interface  requirements  for  modeling  and  optimization  runs;  an  analyst  or 
technician  must  set  up  each  run  separately  and  must  create  a  format  for 
reports.  In  any  future  TDS  development,  the  IOS  needs  to  be  made  more 
user-friendly  with  a  menu  system  of  TDS  options,  and  a  report  generator 
should  be  developed  to  help  users  create  products  to  meet  their  needs. 

Until  such  user-friendly  software  can  be  developed,  a  trained  TDS 
analyst  will  be  needed  to  set  up  and  run  any  type  of  modeling  or 
optimization  problem.  The  models  and  data  files  are  sufficiently  complex 
that  an  untrained  user  would  have  great  difficulty  in  operating  the 
system. 


5.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  TDS  has  considerable  potential  value  for  a  wide  range  of  Air  Force 
decision  makers.  It  can  help  them  to  understand  the  current  jobs  and 
training  programs  of  an  Air  Force  specialty,  to  formulate  plausible 
alternatives,  and  to  test  out  the  consequences  of  proposed  changes.  The 
capabilities  which  the  TDS  provides  should  be  extremely  useful  to  various 
levels  of  Air  Force  managers,  ranging  from  the  Air  Staff  to  training 
managers  at  MAJCOMs,  HQ  ATC,  and  the  Technical  Training  Centers.  The 
system  could  be  effectively  employed  to  provide  objective  information  and 
evaluation  capabilities  for  Utilization  and  Training  Workshops,  Training 
Planning  Teams,  and  other  AFS-oriented  decision  making  groups. 
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5.1  Conceptual  Issues 

5.1.1  A  Systematic  Approach 

One  of  the  major  strengths  of  the  TDS  is  the  systematic  approach  used 
for  development  of  objective  data  bases  that  serve  as  the  foundation  for 
reports  or  briefings  to  appropriate  Air  Force  decision  makers  for  their 
use  in  assessing  the  potential  impact  of  various  proposed  changes  on  an 
Air  Force  specialty.  The  capability  to  project  and  assess  the  consequen¬ 
ces  of  change  has  not  been  available  in  the  past.  Instead,  decision 
makers  had  to  make  subjective  judgments  as  to  possible  outcomes,  based  on 
very  limited  amounts  of  information.  The  TDS  can  provide  substantially 
better  information  for  use  in  such  decisions. 

The  systematic  approach  to  building  Air  Force  specialty- specific  data 
bases — and  collecting  additional  relevant  information  about  AFS  jobs, 
training  programs,  training  costs,  and  possible  alternatives — represents  a 
major  improvement  in  Steps  1  and  2  of  the  ISD  process  (analyze  system 
requirements  and  define  training  requirements;  see  AFR  50-8).  Better 
definition  of  training  requirements  begins  with  an  improved  understanding 
of  the  work  to  be  done  and  how  the  specialty  is  organized  to  accomplish 
such  work. 

5.1.2  Modeling  an  Air  Force  Specialty 

Through  interviews  with  various  Air  Force  managers,  staff  officers, 
and  trainers,  it  became  obvious  that  everyone  had  some  knowledge  of 
various  systems  and  programs,  but  typically  no  one  person  had  a  complete 
picture  of  all  available  training,  nor  all  of  the  jobs,  in  an  Air  Force 
specialty.  Thus  decisions  on  changes  to  the  specialty  currently  were 
being  made,  to  some  degree  at  least,  on  partial  knowledge.  A  key 
improvement  which  could  be  made,  therefore,  is  to  provide  a  more  complete 
picture  of  current  specialty  jobs  and  training. 

TDS  has  focused  on  building  a  specialty  model  that  can  assist  managers 
(and  other  users)  in  understanding  the  flow  of  people  through  initial 
skills  training,  initial  jobs  (including  OJT  and  CDC  completion),  advanced 
training  and  jobs,  PME  and  other  supervisory  and  management  training,  and 
senior-level  responsibilities  and  positions.  The  model  must  depict  the 
multiple  routes  of  entering  the  specialty,  as  well  as  the  multiple  exits 
(cross-training,  separation,  special  assignment,  retirement,  etc.).  The 
model  must  also  accurately  portray  the  various  training  programs  and 
possible  combinations  of  training  which  are  available,  as  well  as  the 
complexity  of  possible  job  assignments.  One  of  the  most  difficult  issues 
is  the  accurate  estimation  of  the  probability  of  being  reassigned  from  one 
type  of  job  to  another;  these  probabilities  directly  and  immediately 
impact  on  the  calculations  of  the  total  training  requirements  of  a 
specialty.  It  is  not  sufficient  for  a  model  to  merely  list  the  available 
training  programs  and  jobs.  A  useful  TDS  model  must  also  portray  the 
dynamics  of  the  reassignment  system  in  order  to  properly  portray  how 
advanced  training  and  OJT  requirements  are  generated  and  met  in  the  USAF. 

A  very  basic  issue  involved  in  creating  such  a  model  is  the  language 
used  to  designate  the  various  jobs  of  a  specialty  and  to  communicate  what 
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such  jobs  involve.  To  describe  each  position  or  job  on  a  task-by-task 
basis  becomes  impossibly  complex,  since  any  specialty  involves  from 
several  hundred  to  over  a  thousand  possible  tasks  which  an  incumbent  might 
be  required  to  perform,  and  these  tasks  are  grouped  many  different  ways  to 
form  jobs.  The  clustering  of  AFS  tasks  into  50  to  100  modules  (Task 
Modules)  provides  the  necessary  simplification  so  that  the  specialty  can 
be  modeled  and  understood.  Naming  the  TMs  in  a  way  that  they  can  be 
adequately  communicated  (among  individuals  at  various  organizational 
levels  and  in  different  functions)  is  a  key  issue.  If  properly  named  by 
experienced  SMEs,  the  set  of  TMs  serves  as  an  excellent  tool  for  assuring 
full  understanding  of  specialty  training  requirements  and  personnel 
utilization  policies. 

Creating  sets  of  TMs  and  modeling  the  jobs  of  specialties  were 
accomplished  quite  successfully  in  the  TDS.  Some  problems  were 
encountered  in  dealing  with  rapidly  changing  conditions.  In  the 
Electronic  Computer  and  Switching  Systems  specialty,  AFS  3  05X4,  for 
example,  some  of  the  jobs  were  being  transferred  to  other  specialties;  new 
shred  outs  were  being  created;  and  training  programs  were  in  transition. 
Another  problem  was  encountered  regarding  proposals  to  merge  various 
specialties  at  advanced  skill  levels  in  order  to  develop  broader  technical 
skills  (RIVET  WORKFORCE).  If  only  one  specialty  of  the  pair  to  be  merged 
was  studied  in  TDS  (as  happened  with  two  specialties),  then  equivalent 
information  was  not  available  for  the  other  AFS  involved.  This  suggests 
that  the  TDS  needs  to  be  expanded  to  deal  with  multiple  career  ladders 
(Mitchell  et  al.,  1987,  section  5.1,  page  110). 

5.1.3  Single  Versus  Multiple  Specialty  Models 

If  major  training  decisions  are  to  be  made  by  multi-specialty 
functional  community  conferences  (such  as  RIVET  WORKFORCE),  then  the 
capability  of  the  TDS  should  be  expanded  to  properly  service  that  need. 
In  such  cases,  the  models  to  be  developed  would  be  for  multiple  related 
specialties  (functional  areas),  rather  than  for  only  one  specialty.  This 
probably  represents  a  new  order  of  complexity.  Work  on  the  present  TDS 
with  the  Security  Police  involved  two  closely  related  specialties 
(Security,  Law  Enforcement)  plus  an  additional  shredout  (Military  Working 
Dog),  and  TDS  was  adapted  to  model  these  fields  (while  tracking  the 
separate  flows).  Ir  this  case,  the  task  list  (OSR)  contained  all  three 
subareas.  For  multiple  specialties  with  separate  task  lists,  some  type  of 
equating  procedure  would  have  to  be  developed,  or  a  new  OA  would  have  to 
be  initiated  with  a  common  task  inventory. 

The  above  discussion  highlights  an  existing  dichotomy  in  the  Air  Force 
training  decision  making  process.  AFS -specific  decisions  can  be  made  in  a 
U  &  TW,  but  multi-AFS  decisions  (which  have  very  significant  training 
implications)  are  made  through  functional  conferences.  Such  different 
decision  making  mechanisms  could  probably  be  merged,  by  using  the  U  &  TW 
process  to  staff  and  implement  major  structural  changes  suggested  by  a 
functional  conference. 

5.1.4  TDS  Versus  Task  Analysis 

Another  philosophical  issue  involves  ISD  techniques  and  technology. 
As  noted  earlier  (Section  2.3.1),  some  individuals  and  organizations 
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interpret  the  ISD  procedures  as  requiring  that  a  complete  task  analysis  of 
every  technical  task  in  the  specialty  be  done  before  any  decision  can 
be  made  as  to  the  training  requirement  (i.e.,  appropriate  place  and  media 
for  training  delivery).  The  basic  philosophy  underlying  TDS,  however,  is 
that  many  training  decisions  can  and  should  be  made  concerning  how  the 
specialty  is  to  be  organized,  the  appropriate  phasing  of  the  delivery  of 
training  (i.e.,  when  in  an  airman's  career),  and  how  tasks  can  be  grouped 
for  training,  before  any  detailed  task  analysis  is  done.  Such  detailed 
task  analysis  can  be  an  extensive,  time-consuming  process  which  is 
labor-intensive  and  highly  variable  in  quality  of  output  (Eschenbrenner, 
DeVries,  Miller,  &  Ruck,  1980).  The  implicit  TDS  philosophy  is  to  provide 
support  for  utilization  and  training  decision  making  in  the  TPT,  U  &  TW,  or 
normal  staffing  process,  and  reserve  detailed  task  analysis  until  after 
some  of  the  critical  decisions  have  been  made. 

Air  Force  guidance  for  training  decision  making  needs  to  clearly 
specify  the  preferred  procedure  for  ISD.  The  recent  revision  of  AFR  50-8, 
Policy  and  Guidance  for  Instructional  Systems  Development  (6  August  1984), 
established  the  TPT  process  as  the  major  vehicle  for  implementing  Air 
Staff  program  management  documents  (PMDs)  and  for  defining  training 
requirements  for  new  weapon  systems.  A  draft  AFR  50-23,  if  approved,  will 
enhance  the  role  of  U  &  TWs  in  making  specialty- specific  training 
decisions.  This  training  decision  philosophy  is  consistent  with  the  type 
of  approach  taken  in  developing  the  TDS.  Perhaps  what  is  now  needed  is  to 
revise  other  ISD  documents,  such  as  AFM  50-2  (25  May  1979)  and  AFP  50-58 
(15  July  1978),  to  reflect  the  new  training  decision  philosophy  and 
current  Air  Force  decision  mechanisms.  Both  the  U  &  TW  and  the  TPT 
mechanisms  for  considering  and  executing  needed  training  changes  were 
developed  after  AFM  50-2  and  AFP  50-58  were  published;  thus,  a  revision  of 
both  documents  to  update  their  guidance  appears  necessary. 


5.2  TDS  Responsibilities 

One  of  the  major  mechanisms  available  for  implementing  the  TDS  as  an 
operational  system  is  a  new  Air  Force  Regulation.  Such  a  regulation  would 
authorize  the  establishment  and  continued  operation  of  the  TDS  and 
delegate  appropriate  responsibilities  to  the  various  Air  Force  agencies 
involved. 

A  proposed  draft  regulation  is  included  as  an  appendix  to  this  report. 
In  recommending  responsibilities  for  various  agencies  in  the  establishment 
and  operation  of  the  system,  this  draft  regulation  follows  recommendations 
made  in  the  TDS  Transition  Plan  (Vaughan  et  al.,  1985). 

The  draft  regulation  embodies  current  training  decision  philosophy  of 
the  ISD  process  as  defined  in  AFR  50-8,  Policy  and  Guidance  for  Instruc¬ 
tional  Systems  Development  (ISD),  6  August  1984.  In  form  and  approach, 
the  draft  parallels  AFR  35-2,  Occupational  Analysis,  23  July  1982.  Since 
the  emphasis  here  is  to  develop  and  provide  training  decision  support 
services  which  are  related  to  but  different  from  the  ISD  and  OA  processes, 
it  is  appropriate  that  TDS  should  be  covered  by  a  separate  regulation. 
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5.3  Recommendations 


5.3.1  An  Operational  TPS 

TDS  should  be  transitioned  into  an  operational  system  through  approval 
of  the  draft  Air  Force  Regulation  included  as  Appendix  A.  Although  there 
is  still  some  work  to  be  done  to  have  a  fully  user-friendly,  automated 
decision  support  system,  TDS  is  now  sufficiently  developed  to  be  of  value 
to  U  &  TWs,  TPTs,  and  other  Air  Force  training  decision  conferences. 

5.3.2  Further  TDS  Testing 

The  present  TDS  was  developed  based  on  information  collected  on  four 
Air  Force  specialties;  the  data  varied  in  quality  and  utility.  TDS  should 
be  extended  by  employing  the  present  techniques  and  software  with  4  to  8 
additional  specialties.  This  would  serve  to  test  the  generalizability  of 
the  system  to  other  types  of  specialties,  as  well  as  to  identify  any 
problems  in  procedures  and  approach  not  yet  detected.  Such  an  extension 
of  the  TDS  R&D  would  be  extremely  valuable,  and  could  be  conducted  by 
AFHRL  in  parallel  with  the  operational  implementation  of  the  system  by  the 
USAFOMC.  Close  coordination  of  these  efforts  will  be  needed. 

5.3.3  Additional  TDS  Research 

A  number  of  further  research  areas  have  been  identified  throughout  the 
TDS  project  and  were  reported  in  the  TDS  Transition  Plan  (Vaughan  et  al., 
1985),  Information  Gathering  report  (Mitchell  et  al.,  1987),  subsystem 
final  reports,  and  earlier  sections  of  this  document.  These  areas  range 
from  basic  research  (training  decay  function  variations  by  type  of 
task  or  AFS,  knowledge  and  skill  requirements  associated  with  groups  of 
tasks  in  an  AFS,  etc.)  to  complex  technological  issues  (generic  or 
AFS-specific  PME  TTMs,  techniques  for  identification  of  representative 
sites,  etc.).  Further  R&D  is  needed  to  adapt  TDS  to  handle  multiple 
specialties  and  for  use  in  the  new  weapon  systems  acquisition  process. 

The  TDS  development  effort  has  successfully  demonstrated  that  such  a 
system  is  practical  and  has  high  potential  utility.  At  the  same  time,  the 
TDS  was  designed  such  that  improved  data  can  be  substituted  without  having 
to  reengineer  the  system.  This  approach  ensures  that  TDS  can  evolve  as 
new  R&D  projects  or  operational  agencies  make  improved  data  available. 
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ACRCNHE  AND  AEEREV3ATICIE 


ABGD 

ABR 

AF 

AFB 

AFCC 

AF/DPPP 

AF/DPFE 

AFHRL 

afhri/id 

AFHRI/MD 

AFHRI/SC 

AF/LE 

AFM 

AFMEA 

AFMPC 

APOSP 

AFP 

AFR 

AES 

ACTS 

ASCII  CODAP 

ATC 

ATC/AC 

ATC/ACMQ 

ATCR 

ATC/TT 

BUMS 

EMT 

CDC 

cars 

CEC 

CODAP 

CONVERT 

CUT 

DCS 

DIAGRAM 

DOD 

ERJ 

EHF 

ENTKYS 

ESBI 

FFMJOB 

FTD 

FUS 


Air  Base  Ground  Defense 

Prefix  for  Airman  Basic  Resident  Courses 

Air  Force  (used  synonymously  with  USAF) 

Air  Force  Base 

Air  Force  Ocmwunications  Command 

Force  Programs,  DCS  Personnel,  HQ  USAF  (formerly  MPPP) 
Training  Programs,  DCS  Personnel  (formerly  MPPT  &  DPPT) 
Air  Force  Human  Resources  Laboratory  (see  also  HRL) 
Training  Systems  Division 
Manpower  &  Personnel  Research  Division 
Information  Sciences  Division  (formerly  the 
Technical  Services  Division) 

DCS  Logistics  &  Engineering,  HQ  USAF 
Air  Force  Manual 

Air  Force  Managment  Engineering  Agency 

Air  Force  Military  Personnel  Center 

Air  Force  Office  of  Security  Police 

Air  Force  Pamphlet 

Air  Force  Regulation 

Air  Force  Specialty 

Advanced  On-the-job  Training  System 

Advanced  CODAP  system 

Air  Training  Command 

DCS  Comptroller,  HQ  ATC 

Oast  &  Management  Analysis,  DCS  Comptroller 
Air  Training  Command  Regulation 
DCS  Technical  Training,  HQ  ATC 

Base  Level  Training  Management  System 
Basic  Military  Training 

Career  Development  Course 
Computer-Delivered  Training  System 

Cost  Estimation  Component  of  the  RCS 
Comprehensive  Occupational  Data  Analysis  Programs 
COCAP  program  to  transpose  case  data  files 
Cross-Utilization  Training 

Deputy  Chief  of  Staff 

CODAP  program  for  displaying  structure  of  groups 
Department  of  Defense 
Direct  Reporting  Unit 

Entity  History  File  in  UTPSIM 
Number  of  people  entering  jobs  or  training  matrix 
in  the  UTPSIM  process  of  the  IUS 
Education  Subject  Block  Index 

Probabilities  of  entities  exiting  jobs  in  UTPSIM 
Field  Training  Detachment 
Field  Utilization  Subsystem 
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GI/CM 

Off 

GROUP 

GRFREL 


HISCRN 

HQ 

HRL 

HRM 

IOS 

ISD 

IVD 

JOBIDS 

JOBTON 

jobtim 

MAC 

MADAR 

MAJCOM 

MODULE 

MODSET 

MPPT 

MPT 

MIS 

MIT 

NOO 

QA 

OLD 

COT 

OMT 

OPR 

ORDB 

OS 

OSD 

OSR 

OUTEHF 


OVRLAP 

BJM 

R© 

IWE 

IMP 

IMS 

POI 

PCM 

PPBS 

FKIMOD 


Ground  launched  Cruise  Missile 
General  Military  Training 

OODAP  program  to  organize  hierarchical  clusters 
Inter-rater  agreement  programs  in  ASCII  CODAP 
(REXALL  in  Fieldata  OODAP) 

History  Screening  program  in  UTPSIM 
Headquarters 

Human  Resources  Laboratory  (also  AFHRL) 

Human  Resources  Management 

Integration  &  Optimization  Subsystem 
Instructional  Systems  Development 
Interactive  Video  Disk 

Titles  and  jab  identification  file  in  UTPSIM 
Job-driven  training  probabilities  file  in  UTPSIM 
Percent  performing  TlMs  per  job  file  in  UTPSIM 

Military  Airlift  Command 
Malfunction  Automatic  Detection,  Analysis 
and  Repair  (AES  328X4  system) 

Major  Command 

Modular  programs  in  ASCII  OODAP 

Create  Module  Factor  Set  program  in  ASCII  OODAP 

Training,  Personnel  Plans,  HQ  USAF  (new  DPPE) 

Manpower,  Personnel,  &  Training 

Military  Training  Standard 

Mobile  Training  Team 

Noncommissioned  Officer 

Occupational  Analysis 
Occupational  Learning  Difficulty 
On-the-Job  Training 

Training  Development  Service  of  the  USAPOMC 
Office  of  Primary  Responsibility 
Occupational  Research  Data  Bank 
Occupational  Survey 
Office  of  the  Secretary  of  Defense 
Occupational  Survey  Report 
CXitput  Entity  History  file  from  UTPSIM 
which  contains  the  job  and  formal  training 
history  of  each  individual  (entity) 

OODAP  program  to  oenpute  coperformance  of  tasks 

Person-Job  Match 

Program  Management  Directive 

Professional  Military  Education 

Program  Management  Plan 

Pipeline  Management  System 

Plan  of  Instruction 

Program  Objective  Memorandum 

Planning,  Programming  &  Budgeting  System 

Print  Modular  Factor  program  in  ASCII  OODAP 
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PIT 


Program  of  Technical  Training 


RCS  Resource/Cost  Subsystem 

R&D  Research  and  Development 

RPR  Request  for  Personnel  Research 

RRC  Resource  Requirements  Component  of  the  RCS 

RUNPAR  Run  Parameter  File  program  of  the  FUS 


SIAM  Simulation  program  tested  for  the  FUS 

SME  Subject-Matter  Expert 

SQA  Separate  Operating  Agency 

SOW  Statement  of  Work 

SP  Security  Police 

STS  Specialty  Training  Standard 


TCC 

TCS 

TD 

TDF 

TDP 

IDS 

TDY 

TE 

T&E 

TIES 

ITMIRN 

TJBTRN 

TM 

TOJOBS 

TPR 

TPT 

TRNPAR 

TRNPRF 

TSO 

IT 

TIC 

TIM 


Training  Capacity  Component  of  the  RCS 

Task  Characteristics  Subsystem 

Task  Difficulty 

Training  Decay  Function 

Training  Development  Plan 

Training  Decisions  System 

Temporary  Duty 

Training  Emphasis 

Ttest  and  Evaluation 

Task  Identification  and  Evaluation  System 
TAIMS-driven  Training  probabilities  in  UTPSIM 
Job  and  TAFMS  Training  matrix  in  UTPSIM 
Task  Module  (new  term  for  Task  Training  Module) 
Probability  matrix  for  entering  AFS  jobs,  UTPSIM 
Trained  Personnel  Requirements 
Training  Planning  Team 

Training  Parameters  file  in  the  UTPSIM  of  the  FUS 
Training  Proficiency  program  in  UTPSIM  used  to 
calculate  CUT  requirements. 

Training  Staff  Officer 
Technical  Training 
Technical  Training  Center 
Task  Training  Module 


UAR  Uniform  Airman  Record 

U&T  Utilization  and  Training 

U&TW  Utilization  and  Training  Workshop 

USAF  United  States  Air  Force 

USAPCMC  USAF  Occupational  Measurement  Center  (ATC) 

USAFCMC/CMD  Occupational  Test  Development  Division 

USAPCMC/CMT  Specialized  Skills  Training  Division  (also  called 

the  Training  Development  Service) 

USAPCMC/GMY  Occupational  Analysis  Division 

UTPSIM  U&T  Pattern  Simulation  Program  in  the  FUS 


XFQSE 


CODAP  program  to  transpose  case  data  files 
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DEPARTMENT  OF  THE  AIR  FORCE  CRAFT 

Headquarters  US  Air  Force 
Washington  DC  20330 


AF  REGULATION  50-XX 


Training 

POLICY  AND  GUIDANCE  FOR  THE  TRAINING  DECISIONS  SYSTEM 


This  regulation  authorizes  the  Air  Force  Training  Decisions  System  (TDS) , 
outlines  its  functions,  and  delegates  responsibilities  for  its  operation 
and  maintenance.  The  TDS  provides  necessary  objective  data  and  decision 
support  services  for  Air  Force  activities  involved  in  making  major  training 
decisions.  Supplements  must  have  prior  approval  (AFR  5-13) . 


1.  TDS  Explained.  The  Air  Force  TDS  is  a  computer-based,  decision 
support  system  which  provides  services  to  assist  Air  Force  training 
decision  making  agencies  or  conferences  in  analyzing  AF  Specialty  (AFS)  or 
new  weapons  system  operations  and  defining  AFS  education  and  training 
requirements  (Steps  1  and  2  of  the  ISD  process;  see  AER  50-8) .  The 
objective  of  the  TDS  is  to  collect  empirical  data  concerning  personnel 
utilization  and  training  patterns,  associated  training  requirements,  and 
costs  of  training  for  a  specialty  (or  group  of  specialties) ,  and  to 
analyze,  process,  and  summarize  such  data  into  forms  useful  to  AF  training 
decisionmakers.  Using  TDS  data,  plans  or  guidance  on  expected  changes  in 
an  AFS,  and  relevant  input  from  other  AF  agencies,  decisionmaking  bodies 
such  as  multicammand  Utilization  and  Training  Workshops  (U&TWs) ,  Training 
Planning  Teams  (TPTs)  and  other  training  decision  conferences  will  optimize 
their  decisions  in  terms  of  training  relevance,  practicality,  and  costs. 

NOTE:  Other  elements  of  the  ISD  process  are  outlined  in  AFR  50-8,  AFMs  50-2 
and  50-62,  and  AFP  50-58.  Interservice  Procedures  for  ISD  may  be  used  in 
lieu  of  or  in  addition  to  AFP  50-58. 


New  Regulation 
No.  of  Printed  Pages:  3 
OFR:  DFPIS  (LIC  J.  Jasper) 
Approved  by:  LbGen  T.  Hickey 

Writer-Editor:  _ 

Distribution:  F 
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2.  HOW  TDS  IS  TO  EE  USED: 


a.  In  support  of  U&TWs,  TPTs  and  other  AFS  training  decision  meetings, 
the  TDS  will: 

(1)  Describe  the  AFS  in  terms  of  the  groups  of  tasks  which  could 
or  should  be  trained  together  (Task  Modules  or  TMs)  using  the  relevant 
occupational  analysis  (see  AFR  35-2)  as  a  starting  point,  and  analyze  the 
characteristics  of  AFS  tasks  or  TMs,  the  way  in  which  the  work  of  the  AFS 
is  organized  (i.e.,  the  jobs  which  are  performed) ,  specialty  training 
programs  (in  terms  of  tasks  or  TMs) ,  and  the  dynamic  flow  of  people 
through  both  jobs  and  training  programs  over  the  span  of  their  careers. 
This  qualitative  description  of  the  AFS  provides  the  specialty  overview 
and  serves  as  the  baseline  for  managment  decisions  concerning  AFS 
training. 

(2)  Collect  quantitative  data  concerning  how  training  is  and 
might  be  allocated  to  various  training  settings  (classroom,  self-study, 
field  training,  on-the-job  training,  etc.),  and  develop  statistical 
functions  to  relate  hours  of  training  to  required  proficiency  for  each 
setting. 

(3)  Estimate  transition  probabilities  among  various  training 
programs  and  jobs  for  successive  assignments  so  as  to  model,  in 
quantitative  terms,  the  utilization  and  training  pattern  describing  the 
AFS,  outlined  in  (1)  above. 

(4)  Develop  realistic  alternative  patterns  or  AFS  models,  based 
on  programmed  or  desired  AFS  changes,  to  provide  a  basis  for  comparative 
evaluation  among  possible  choices  which  the  U&TW  or  other  AFS  training 
decision  agency  might  recommend. 

(5)  Develop  data  bases  reflecting  manpower,  resources,  and  annual 
recurring  costs  of  required  training  based  on  actual  data  from  represen¬ 
tative  sites  and  training  programs,  with  predictions  of  total  requirements 
for  the  current  and  alternative  AFS  models  being  examined,  to  provide  the 
basis  for  evaluating  the  relative  merit  of  each  alternative. 

(6)  Provide  statistical  analysis  services  including  optimization 
routines  and  summary  reports  as  directed  by  the  U&TW  or  other  AFS  training 
decision  body  to  assist  in  their  evaluation  and  decision  processes. 

(7)  Maintain  and  update  TDS  data  bases  for  an  AFS,  as  directed  by 
the  U&TW  or  other  AFS  training  decision  body  for  the  lifecycle  of  the 
training  review  process. 
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b.  U&TWs  and  other  AFS  training  decision  meetings  will  use  the  TDS  to: 

(1)  Formulate  a  description  of  the  specialty  or  specialties  as  a 
basis  for  analysis  of  AES  training  requirements. 

(2)  Examine  alternative  job  structures  (utilization  and  training 
patterns)  based  on  known  or  desired  changes  in  the  AFS. 

(3)  Evaluate  the  manpower,  resources,  and  cost  implications  of  the 
various  alternatives. 

(4)  Recommend  adoption  of  the  AES  Utilization  and  Training  pattern 
which  optimizes  AFS  training  in  terms  of  managers'  preferences  or  total 
training  costs. 

(5)  Formulate  an  action  plan  to  implement  the  selected  training 

options. 

3.  HQ  USAF  RESPONSIBILITIES : 

a.  General: 

(1)  Promote  the  use  of  TDS  services  by  U&TWs  and  other  training 
decisionmaking  agencies. 

(2)  Coordinate  actions  relevant  to  the  TDS  among  HQ  USAF,  MAJCOM, 
SOA,  and  DRU  offices  of  primary  and  collateral  training  responsibility. 

(3)  Provide  staff  assistance  to  TDS  analysts  collecting 
information  on  functional  developments  and  plans  as  they  inpact  specific 
AFSs  and  AFS  training  programs,  and  provide  preferences  among  alternative 
U&T  patterns  for  TDS  and  U&TW  analysis. 

b.  HQ  USAF/DPPT  will  use  TDS  services  in  planning  and  executing  the 
management  of  Air  Force  training  programs.  Specifically,  HQ  USAF/DPPT  has 
responsibility  to: 

(1)  Establish  Air  Force  policy  for  the  creation,  operation,  and 
maintenance  of  the  TDS. 

(2)  Monitor  the  use  of  TDS  services  throughout  the  Air  Force. 

(3)  Review  and  approve  U&TW  action  plans  to  insure  that 
appropriate  uses  of  TDS  services  have  been  made. 

(4)  Provide  for  the  continued  development  and  refinement  of  the 
TDS  through  sponsorship  of  appropriate  Requests  for  Personnel  Research 
(RFRs)  and  associated  funding  priority  via  the  Planning,  Programming,  and 
Budgeting  System  (PPBS) . 


4.  MAIOOM,  SOA,  and  EMJ  Responsibilities: 

a.  Functional  Managers  will: 

(1)  Participate  in  U&TWs  and  other  training  decision  meetings 
using  TDS  data  to  support  critical  training  decisions  for  relevant  AESs. 

(2)  Coordinate  and  support  visit  requests  of  TDS  analysts  (AFHRL, 
USAFOMC,  or  contractors)  to  Headquarters  or  operational  units  to  collect 
training  data,  as  needed. 

(3)  Coordinate  and  support  requests  of  TDS  analysts  for  resource, 
manpower,  and  cost  data  for  training  programs  within  the  command. 

(4)  Provide  AFS  subject  matter  experts  (SMEs)  from  subordinate 
representative  units  for  TDS  data  collection  and  validation  meetings,  as 
required. 

(5)  Review  and  staff  completed  TDS  studies  and  U&TWs  for  relevant 
AFSs  to  assess  potential  impact  on  MAJOCM,  SOA,  or  EMJ  operations. 

b.  ATC  Additional  Responsibilities: 

(1)  Prepare  Air  Force  numbered  TDS  publications  and  amend  the  ISD 
literature  to  include  TDS  as  directed  by  HQ  USAF/DPPT. 

(2)  Serve  as  the  Air  Force  clearinghouse  for  TDS  information  and 
support  other  MAJOQMS,  SOA s,  and  EMJs  in  obtaining  TDS  related  data. 

(3)  Develop  and  maintain  guidance  for  the  use  of  TDS  services  by 
chairpersons  of  U&TWs. 

(4)  Provide  guidance,  direction,  and  staff  surveilance  for  the 
operation  of  the  TDS. 

(5)  Program  for  required  manpower  and  funds  to  establish  and 
maintain  the  TDS  within  the  USAFOMC. 

(6)  Provide  internal  coordination  of  U&TWs  with  staff  and 
subordinate  units  to  assess  the  inpact  of  such  plans  on  relevant  training 
documents  and  plans. 

(7)  Provide  access  of  TDS  analysts  to  HQ  ATC  cost  analysis  and 
training  resources  databases  for  collection  of  AFS  training  cost  data. 

(8)  Provide  access  for  TDS  analysts  to  the  IMS  database  for  AFS 
training  input,  attrition,  and  production  figures  (current  and 
historical) . 

(9)  Coordinate  with  AFHRL  to  define  and  support  RTRs  for  the 
further  development  of  the  system. 


c.  USAPOMC  Responsibilities: 

(1)  Establish  the  TDS  as  an  operational  capability  through 

installation  of  TDS  software  on  the  USAPCMC  computer,  and  maintain  a 
hardline  interface  with  the  AFHRL  computer  to  provide  access  to  the  TDS, 
CODAP,  and  other  support  software  or  files.  The  system  shall,  at  a 

minimum,  have  the  capabilities  outlined  in  paragraph  2.  above. 

(2)  Designate  and  train  selected  analysts  to  serve  as  TDS 

specialists  responsible  for  conducting  TDS  projects. 

(3)  Solicit  and  consider  candidate  AFSs  for  TDS  studies,  and 

submit  such  candidates  for  prioritization  along  with  occupational  analysis 
and  training  development  projects. 

(4)  Schedule  approved  TDS  studies  in  an  appropriate  section  of 
the  USAF  PIT,  Part  2. 

(5)  Program  and  budget  for  travel  by  analysts  in  support  of 
priority  TDS  projects  and  coordinate  such  travel  with  relevant 
organization!  units.  Direct  communication  with  AF  units  is  authorized, 
with  information  copy  to  relevant  MAJGCM,  SOA,  or  DRD  Headquarters 
training  or  functional  staff. 

(6)  Convene  meetings  and  AES  panels  as  required  to  develop, 

review,  or  validate  TDS  data  files  and  major  findings. 

(7)  Prepare  a  report  of  each  study  for  submission  to  the  U&TW  or 
other  training  decision  making  meeting  and,  with  the  approval  of  the  U&TW, 
for  publication. 

(8)  Maintain  and  update  TDS  studies  and  data  files  as  needed. 

(9)  Coordinate  with  AFHRL  on  the  maintenance  and  further 
development  of  the  TDS,  to  insure  that  TDS  software  and  procedures  are 
refined  to  resolve  any  problems  encountered  in  the  operational  use  of  TDS. 

d.  AFHRL  Responsibilities: 

(1)  Serve  as  the  focal  point  for  further  TDS  research  and 
development;  maintain  and  refine  TDS  software  and  procedures  on  the  basis 
of  user  feedback. 

(2)  Participate  in  a  representative  sample  of  Air  Force  U&TW 
conferences  to  assess  the  need  and  form  of  future  TDS  R&D  needs. 

(3)  Develop  R&D  projects  to  improve  the  system  and  enhance  its 
utility  to  the  Air  Force. 

(4)  Provide  representation  to  serve  on  TDS  project  priority 
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review  panels  and  coordinate  on  AFHRL  assistance  needed  for  such  projects. 

(5)  Provide  coordination  with  ASCII  CODAP  development  to  insure 
the  systems  remain  compatible  so  as  to  facilitate  and  optimize  the  use  of 
CODAP  data  in  TDS  projects. 

(6)  Conduct  research  to  automate  the  flew  of  information  between 
TDS  users  and  additional  potential  users  (Manpower,  Personnel,  and 
Training  data  systems) . 

(7)  Coordinate  research  efforts  to  maximize  the  cross  utilization 
of  R&D  results  among  developing  MPT  systems. 


BY  ORDER  OF  THE  SECRETARY  OF  THE  AIR  FORCE 
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LARRY  D.  WELCH,  General,  USAF 
Chief  of  Staff 


JAMES  H.  DELANEY,  Colonel,  USAF 
Director  of  Administration 
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